[1912.08648] Inferring the causal effect of journals on citations

Abstract:  Articles in high-impact journals are by definition more highly cited on average. But are they cited more often because the articles are somehow “better”? Or are they cited more often simply because they appeared in a high-impact journal? Although some evidence suggests the latter the causal relationship is not clear. We here compare citations of published journal articles to citations of their preprint versions to uncover the causal mechanism. We build on an earlier model to infer the causal effect of journals on citations. We find evidence for both effects. We show that high-impact journals seem to select articles that tend to attract more citations. At the same time, we find that high-impact journals augment the citation rate of published articles. Our results yield a deeper understanding of the role of journals in the research system. The use of journal metrics in research evaluation has been increasingly criticised in recent years and article-level citations are sometimes suggested as an alternative. Our results show that removing impact factors from evaluation does not negate the influence of journals. This insight has important implications for changing practices of research evaluation.

 

PSI | OA Metrics Generator

“This new tool from PSI generates open access usage statistics for publishers and university repositories. Partnership with Scholarly IQ ensures these metrics can be fully COUNTER compliant. With PSI’s OA Metrics Generator you can develop a deeper understanding of your organisational readership….”

Further improvement of our metrics—will plan S affect them?: Upsala Journal of Medical Sciences: Vol 124, No 4

“What impact could these changes have for our journal? Quite likely, forcing researchers funded by public grants provided by national and European research councils to publish under open access policies would benefit our journal. Upsala Journal of Medical Sciences has applied gold Open Access for the last 10?years, i.e. all published articles have been fully accessible immediately upon release without a publication paywall. It is most likely that this feature of our journal has enhanced our performance substantially. When investigating the influence of the open access publishing format on the IF values of open access and closed access journals over a 5-year period, we found no obvious effect on the journals chosen for the review (4). In order to have a longer time perspective, we looked at the same journals 5?years later when the IF values for 2018 were released (Tables 1 and 2). Interestingly, there was no obvious increase of the values during this 10-year period. This was despite a remarkable increase of both the number of journals and articles. But, again, no differences between the two groups of journals were discernible. It is worthy of note that two Uppsala-based journals, one in the closed category (Amyloid; more than two-fold) and one in the open access (UJMS; almost four-fold) display quite obvious improvements. So, in essence we have nothing to fear. Open access publishing will most probably slowly increase, and fees will have to be paid in some way for the service publishers provide. APCs will become a more common phenomenon, and the level will depend on the quality and reputation characterising the journal. At present the board of our society has no plans for a change of our no-APC policy. Of course, such an offer attracts many contributions from research environments with meagre economic resources. It also constitutes a substantial work load for the editorial board. There is only room for some 40–50 published papers a year, and, with submission figures close to 300, there is a fairly high rejection rate. From the editors’ point of view, we have to attract more papers good enough for acceptance. And we take this opportunity to repeat that we offer a fast track for high-quality papers….”

AN OPEN LETTER FOLLOWING THE CONSULTATION ON TRANSFORMATIVE JOURNALS

“While we are supportive of the vast majority of the criteria proposed in the consultation, we are concerned that, if the transformative journal concept as envisaged by the cOAlition is applied in full, Transformative Journals will not deliver the full transition we believe is possible. We feel duty bound, having proposed this approach, to share our concerns with you now in an open way.

1. The timelines proposed and the rates of OA transition are unworkable and could be counterproductive In our earlier responses to Plan S, we repeatedly and publicly committed ourselves to transitioning all of our journals, our hybrid portfolio of 1900 journals (Springer Nature-owned and society-owned) along with Nature itself and all other Nature-branded journals, to immediate, full OA for all primary research and we will do everything we can to make this a sustainable reality in the shortest possible time. But the speed by which this can happen is not solely in our hands; it is also hugely dependent on the rate at which other funders, institutions and consortia commit to supporting Gold OA, as a zero embargo green OA approach will undermine the sustainability of journals as they transition and hamper the move to open science. …

2. The waiver requirements are unsustainable 

At Springer Nature we have established waiver policies already in place6 for researchers unable to access APC funding and for those authors based in the world’s lowest income countries as defined by the World Bank. As the largest OA publisher we have given more waivers than anyone else. For obvious reasons, this applies only to authors seeking to publish in one of our 600 fully OA journals. For authors without OA funding and seeking to publish in one of our other journals, they are able to do so for free via the subscription route….

We propose the below as an alternative timeframe and workable set of metrics:

1. Year-on-year growth of OA content at the same rate as the increase in global research supported by funders and institutions committed to funding Gold OA.

2. Journals to be flipped when OA content reaches 90%.

3. Progress to be reviewed in 2024, as per cOAlition decision to review progress more widely, and commitments adapted accordingly then in light of progress to date….”

Alternative conditions needed in order for cOAlition S’s proposal for Transformative Journals to succeed | Group | Springer Nature

“As the largest OA publisher, and the publisher that first floated the idea of Transformative Journals, Springer Nature is appealing to cOAlition S in an open letter not to lose the opportunity Transformative Journals offer to speed up the transition to OA. Unless changes are made to the conditions being proposed the publisher believes it would be unable to commit to its journals participating.

Commenting, Steven Inchcoombe, Chief Publishing Officer Springer Nature, said:

“Springer Nature remains committed to moving more quickly towards OA and is proud to be the publisher of almost a quarter of all fully OA articles ever published. We first floated the idea of Transformative Journals in May as we believed that, by harnessing the investment, track record, editorial expertise, and the trust the research communities have in these long-standing journals, the transition to OA could be significantly accelerated and enable many of these journals, including highly selective ones such as Nature, to get on the path to OA.”

“We are concerned, however, that the thresholds proposed by cOAlition S could have unintended consequences. Authors of research funded by cOAlition S members are likely to see their journal choice severely restricted, organisations committed to OA could see a doubling of the content they need to fund, and ultimately many journals may have to rule themselves out, resulting in a slowdown of the very transition we both want to see.”

While Springer Nature is supportive of the vast majority of the criteria proposed in the consultation, the company has significant concerns regarding:

the proposed timelines and metrics which would place conditions on publishers to not only grow OA content at a faster rate than the growth of funders willing to fund immediate OA but to do this at a time when the global share of cOAlition S funded research is effectively declining, and
the requirements for waivers which would see much more research published for free, undermining the sustainability of those journals. This could lead to those organisations that are committed to Gold OA having to support twice the content they were funding at the point of the flip, which is not fair, reasonable or sustainable….”

Good Practices – Research Institutes – DORA

“DORA’s ultimate aim is not to accumulate signatures but to promote real change in research assessment.  One of the keys to this is the development of robust and time-efficient ways of evaluating research and researchers that do not rely on journal impact factors. We are keen to gather and share existing examples of good practice in research assessment, including approaches to funding and fellowships, hiring and promotion, and awarding prizes, that emphasize research itself and not where it is published. 

If you know of exemplary research assessment methods that could provide inspiration and ideas for research institutes, funders, journals, professional societies, or researchers, please contact DORA….”

To fix research assessment, swap slogans for definitions

“Two years ago, the DORA steering committee hired me to survey practices in research assessment and promote the best ones. Other efforts have similar goals. These include the Leiden Manifesto and the HuMetricsHSS Initiative.

My view is that most assessment guidelines permit sliding standards: instead of clearly defined terms, they give us feel-good slogans that lack any fixed meaning. Facing the problem will get us much of the way towards a solution.

Broad language increases room for misinterpretation. ‘High impact’ can be code for where research is published. Or it can mean the effect that research has had on its field, or on society locally or globally — often very different things. Yet confusion is the least of the problems. Descriptors such as ‘world-class’ and ‘excellent’ allow assessors to vary comparisons depending on whose work they are assessing. Academia cannot be a meritocracy if standards change depending on whom we are evaluating. Unconscious bias associated with factors such as a researcher’s gender, ethnic origin and social background helps to perpetuate the status quo. It was only with double-blind review of research proposals that women finally got fair access to the Hubble Space Telescope. Research suggests that using words such as ‘excellence’ in the criteria for grants, awards and promotion can contribute to hypercompetition, in part through the ‘Matthew effect’, in which recognition and resources flow mainly to those who have already received them….”

Addendum to the cOAlition S Guidance on the Implementation of Plan S | Plan S

“cOAlition S endorse a number of strategies to encourage subscription publishers to transition to Open Access. These approaches are referred to as ’transformative arrangements’ and include transformative agreements, transformative model agreements and transformative journals[1].

The Guidance on the Implementation of Plan S indicates an ambition of developing a framework for ‘transformative journals’. Such ‘transformative journals’ are journals that (i) gradually increase the share of Open Access content, (ii) offset subscription income from payments for publishing services (to avoid double payments), and (iii) have a clear commitment to a transition to full and immediate Open Access for all peer-reviewed scholarly articles within an agreed timeframe.

The requirements below constitute this framework.

[Here omitting 8 mandatory criteria for transformative journals and 3 suggested criteria.]

We are now seeking input from the community on this draft framework and encourage all interested stakeholders to respond. The consultation on this draft framework is open until 09.00 CET on Monday 6th January 2020. We plan to publish a final version of this framework by the end of March 2020.”

Addendum to the cOAlition S Guidance on the Implementation of Plan S | Plan S

“cOAlition S endorse a number of strategies to encourage subscription publishers to transition to Open Access. These approaches are referred to as ’transformative arrangements’ and include transformative agreements, transformative model agreements and transformative journals[1].

The Guidance on the Implementation of Plan S indicates an ambition of developing a framework for ‘transformative journals’. Such ‘transformative journals’ are journals that (i) gradually increase the share of Open Access content, (ii) offset subscription income from payments for publishing services (to avoid double payments), and (iii) have a clear commitment to a transition to full and immediate Open Access for all peer-reviewed scholarly articles within an agreed timeframe.

The requirements below constitute this framework.

[Here omitting 8 mandatory criteria for transformative journals and 3 suggested criteria.]

We are now seeking input from the community on this draft framework and encourage all interested stakeholders to respond. The consultation on this draft framework is open until 09.00 CET on Monday 6th January 2020. We plan to publish a final version of this framework by the end of March 2020.”

The fundamental problem blocking open access and how to overcome it: the BitViews project

Abstract:  In our view the fundamental obstacle to open access (OA) is the lack of any incentive-based mechanism that unbundles authors’ accepted manuscripts (AMs) from articles (VoRs). The former can be seen as the public good that ought to be openly accessible, whereas the latter is owned by publishers and rightly paywall-restricted. We propose one such mechanism to overcome this obstacle: BitViews. BitViews is a blockchain-based application that aims to revolutionize the OA publishing ecosystem. Currently, the main academic currency of value is the citation. There have been attempts in the past to create a second currency whose measure is the online usage of research materials (e.g. PIRUS). However, these have failed due to two problems. Firstly, it has been impossible to find a single agency willing to co-ordinate and fund the validation and collation of global online usage data. Secondly, online usage metrics have lacked transparency in how they filter non-human online activity. BitViews is a novel solution which uses blockchain technology to bypass both problems: online AMS usage will be recorded on a public, distributed ledger, obviating the need for a central responsible agency, and the rules governing activity-filtering will be part of the open-source BitViews blockchain application, creating complete transparency. Once online AMS usage has measurable value, researchers will be incentivized to promote and disseminate AMs. This will fundamentally re-orient the academic publishing ecosystem. A key feature of BitViews is that its success (or failure) is wholly and exclusively in the hands of the worldwide community of university and research libraries, as we suggest that it ought to be financed by conditional crowdfunding, whereby the actual financial commitment of each contributing library depends on the total amount raised. If the financing target is not reached, then all contributions are returned in full and if the target is over-fulfilled, then the surplus is returned pro rata.