Publishing computational research – a review of infrastructures for reproducible and transparent scholarly communication | Research Integrity and Peer Review | Full Text

Abstract:  Background

The trend toward open science increases the pressure on authors to provide access to the source code and data they used to compute the results reported in their scientific papers. Since sharing materials reproducibly is challenging, several projects have developed solutions to support the release of executable analyses alongside articles.

Methods

We reviewed 11 applications that can assist researchers in adhering to reproducibility principles. The applications were found through a literature search and interactions with the reproducible research community. An application was included in our analysis if it (i) was actively maintained at the time the data for this paper was collected, (ii) supports the publication of executable code and data, (iii) is connected to the scholarly publication process. By investigating the software documentation and published articles, we compared the applications across 19 criteria, such as deployment options and features that support authors in creating and readers in studying executable papers.

Results

From the 11 applications, eight allow publishers to self-host the system for free, whereas three provide paid services. Authors can submit an executable analysis using Jupyter Notebooks or R Markdown documents (10 applications support these formats). All approaches provide features to assist readers in studying the materials, e.g., one-click reproducible results or tools for manipulating the analysis parameters. Six applications allow for modifying materials after publication.

Conclusions

The applications support authors to publish reproducible research predominantly with literate programming. Concerning readers, most applications provide user interfaces to inspect and manipulate the computational analysis. The next step is to investigate the gaps identified in this review, such as the costs publishers have to expect when hosting an application, the consideration of sensitive data, and impacts on the review process.

ACM Signs New Open Access Agreements with Four Leading Universities | MIT Libraries News

“ACM, the Association for Computing Machinery, entered into transformative open access agreements with several of its largest institutional customers, including the University of California (UC), Carnegie Mellon University (CMU), Massachusetts Institute of Technology (MIT), and Iowa State University (ISU). The agreements, which run for three-year terms beginning January 1, 2020, cover both access to and open access publication in ACM’s journals, proceedings and magazines for these universities, and represent the first transformative open access agreements for ACM….”

About ACM’s Decision to Sign Letters Regarding OSTP’s Proposal to Mandate Zero Embargo of Research Articles

“There have been some strong reactions to ACM’s decision to sign on to letters to the White House Office of Science and Technology Policy (OSTP) as a response to a new directive that OSTP is preparing to issue. That directive would eliminate the current 12-month embargo period for opening U.S. federally funded research publications.

ACM both supports and enables open access models and has worked to support a long and growing list of open access initiatives (see https://www.acm.org/publications/openaccess), doing so in a responsible and sustainable way. For the past decade, all ACM authors have had the right to post accepted versions of their articles in pre-print servers, personal websites, funder websites, and institutional repositories with a zero embargo. More recently, for example, ACM has introduced the OpenTOC service that enables free full-text downloads from links on conference websites immediately upon publication.

It is important to understand why ACM opted to sign the letters opposed to the OSTP zero embargo directive. A long dialogue between OSTP and scholarly publishers led to broad agreement on the current policy (from 2013) of a 12-month embargo for digital libraries. However, due process was not followed for the proposed change to zero embargo. The new directive fails to take into account the significant progress that has been made by ACM and other societies with respect to open access publication since 2013 and there was no dialogue with stakeholders prior to proposing the change.”

Free Machine Learning Repository Increases Accessibility in Genome Research | Technology Networks

Although the importance of machine learning methods in genome research has grown steadily in recent years, researchers have often had to resort to using obsolete software. Scientists in clinical research often did not have access to the most recent models. This will change with the new free open access repository: Kipoi.

Kipoi enables an easy exchange of machine learning models in the field of genome research. The repository was created by Julien Gagneur, Assistant Professor of Computational Biology at the TUM, in collaboration with researchers from the University of Cambridge, Stanford University, the European Bioinformatics Institute (EMBL-EBI) and the European Molecular Biology Laboratory (EMBL)….”

arXiv Update – January 2019 – arXiv public wiki – Dashboard

“In 2018, the repository received 140,616 new submissions, a 14% increase from 2017. The subject distribution is evolving as Computer Science represented about 26% of overall submissions, and Math 24%. There were about 228 million downloads from all over the world. arXiv is truly a global resource, with almost 90% of supporting funds coming from sources other than Cornell and 70% of institutional use coming from countries other than the U.S….”

Could This Search Engine Save Your Life? – The Chronicle of Higher Education

One of the Allen Institute’s priorities is an academically oriented search engine, established in 2015, called Semantic Scholar (slogan: “Cut through the clutter”). The need is great, with more than 34,000 peer-reviewed journals publishing 2.5 million articles a year. “What if a cure for an intractable cancer is hidden within the tedious reports on thousands of clinical studies?,” Etzioni once said.

Although Semantic Scholar has focused so far on computer and biomedical sciences, Etzioni says that the engine will soon push into the social sciences and the humanities as well. The Chronicle spoke with him about information overload, impact factors’ imperfect inevitability, and the promise and perils of AI….”

Open Access for Impact: How Michael Nielsen Reached 3.5M Readers – SPARC

“Michael Nielsen recognizes that Open Access is often argued about in the abstract.  To help the discussion move from the conceptual to the concrete, he recently decided to openly share his experience of writing an open-access book, “Neural Networks and Deep Learning” http://neuralnetworksanddeeplearning.com/chap1.html  to illustrate the positive impact and far reach of online publishing….”

Studies on Subject-Specific Requirements for Open Access Infrastructure

Abstract:  The main purpose of this chapter is to report how researchers investigating in the area of e-Infrastructures organize their activities of “data and publication management” and themselves rely on research infrastructures to do so. Due to the early age of this field and its rather multidisciplinary computer science character, no well-established research infrastructure is available and researchers tend to follow “infrastructure-flavoured” solutions local to their organizations. As a consequence, the authors of this chapter (from the DLib research group at CNR, Italy and the MADGIK research group at the University of Athens, Greece) opted to approach this study by collecting a number of experiences from relevant stakeholders in the field in order to identify “local infrastructure” commonalities and “research infrastructure” desiderata.

Welcome to Cogprints – Cogprints

“Welcome to CogPrints, an electronic archive for self-archive papers in any area of Psychology, Neuroscience, and Linguistics, and many areas of Computer Science (e.g., artificial intelligence, robotics, vison, learning, speech, neural networks), Philosophy (e.g., mind, language, knowledge, science, logic), Biology (e.g., ethology, behavioral ecology, sociobiology, behaviour genetics, evolutionary theory), Medicine (e.g., Psychiatry, Neurology, human genetics, Imaging), Anthropology (e.g., primatology, cognitive ethnology, archeology, paleontology), as well as any other portions of the physical, social and mathematical sciences that are pertinent to the study of cognition….”