[1912.08648] Inferring the causal effect of journals on citations

Abstract:  Articles in high-impact journals are by definition more highly cited on average. But are they cited more often because the articles are somehow “better”? Or are they cited more often simply because they appeared in a high-impact journal? Although some evidence suggests the latter the causal relationship is not clear. We here compare citations of published journal articles to citations of their preprint versions to uncover the causal mechanism. We build on an earlier model to infer the causal effect of journals on citations. We find evidence for both effects. We show that high-impact journals seem to select articles that tend to attract more citations. At the same time, we find that high-impact journals augment the citation rate of published articles. Our results yield a deeper understanding of the role of journals in the research system. The use of journal metrics in research evaluation has been increasingly criticised in recent years and article-level citations are sometimes suggested as an alternative. Our results show that removing impact factors from evaluation does not negate the influence of journals. This insight has important implications for changing practices of research evaluation.

 

Further improvement of our metrics—will plan S affect them?: Upsala Journal of Medical Sciences: Vol 124, No 4

“What impact could these changes have for our journal? Quite likely, forcing researchers funded by public grants provided by national and European research councils to publish under open access policies would benefit our journal. Upsala Journal of Medical Sciences has applied gold Open Access for the last 10?years, i.e. all published articles have been fully accessible immediately upon release without a publication paywall. It is most likely that this feature of our journal has enhanced our performance substantially. When investigating the influence of the open access publishing format on the IF values of open access and closed access journals over a 5-year period, we found no obvious effect on the journals chosen for the review (4). In order to have a longer time perspective, we looked at the same journals 5?years later when the IF values for 2018 were released (Tables 1 and 2). Interestingly, there was no obvious increase of the values during this 10-year period. This was despite a remarkable increase of both the number of journals and articles. But, again, no differences between the two groups of journals were discernible. It is worthy of note that two Uppsala-based journals, one in the closed category (Amyloid; more than two-fold) and one in the open access (UJMS; almost four-fold) display quite obvious improvements. So, in essence we have nothing to fear. Open access publishing will most probably slowly increase, and fees will have to be paid in some way for the service publishers provide. APCs will become a more common phenomenon, and the level will depend on the quality and reputation characterising the journal. At present the board of our society has no plans for a change of our no-APC policy. Of course, such an offer attracts many contributions from research environments with meagre economic resources. It also constitutes a substantial work load for the editorial board. There is only room for some 40–50 published papers a year, and, with submission figures close to 300, there is a fairly high rejection rate. From the editors’ point of view, we have to attract more papers good enough for acceptance. And we take this opportunity to repeat that we offer a fast track for high-quality papers….”

Good Practices – Research Institutes – DORA

“DORA’s ultimate aim is not to accumulate signatures but to promote real change in research assessment.  One of the keys to this is the development of robust and time-efficient ways of evaluating research and researchers that do not rely on journal impact factors. We are keen to gather and share existing examples of good practice in research assessment, including approaches to funding and fellowships, hiring and promotion, and awarding prizes, that emphasize research itself and not where it is published. 

If you know of exemplary research assessment methods that could provide inspiration and ideas for research institutes, funders, journals, professional societies, or researchers, please contact DORA….”

Dutch universities and research funders move away from the impact factor – ScienceGuide

“By the end of 2019, all parties involved in this project pledge to have signed DORA  . This commitment has to be more than an empty gesture. For example, norms such as ‘four publications to obtain a PhD’ will be abolished, and NWO and ZonMw will no longer inquire about h-index or journal impact factor when academics submit grant applications. Instead of asking for a publication list and CV, they will ask for a more ‘narrative’ approach – inquiring about why this research is important, and why the applicant is the right person to carry it out.

This change will be fast, but not instant. The parties involved acknowledge that change takes time. Considering that to focus metrics such as impact factors took decades to become part of established practices, unlearning these routines will require a considerable amount of time, energy and perseverance. Correctly identifying diverse forms of talent and ‘good research’ will be a learning experience: “To accelerate the desired cultural change in recognition and rewards, we at NWO and ZonMW will strongly focus on training and instruction for our grant evaluation committees.” …”

A new era for research publication: Will Open Access become the norm? – Hotta – – Journal of Diabetes Investigation – Wiley Online Library

“This new challenge [Plan S] causes some concerns to us. This program is unlikely to be equivalent between Europe and the United States8). because key US federal agencies such as National Institute of Health (NIH), mandate a ‘green’ Open Access policy, whereby articles in subscription journals are automatically made available after a 12-month embargo. This policy protects the existing ‘paywalled’ subscription business model. Also, ‘Plan S’ does not allow for scientists to publish their papers in hybrid journals….

One piece of bright news, however, is that Open Access publication fees would be covered by funders or research institutions, not by individual researchers. Although our journal is already Open Access, we have some concerns regarding the publication fee being covered by either researchers or institutions….”

Given that the publishing industry is approaching a new era in which 85% or more of journals are Open Access, it is necessary for us to develop a survival strategy against this coming fierce competition….

The effect of “open access” on journal impact factors: A causal analysis of medical journals – NASA/ADS

Abstract:  The Journal Impact Factor (JIF) has a significant influence on authors of research paper submissions. Whether open access (OA) is beneficial to JIFs and the dissemination of academic research results remains an under-examined issue. Using panel data analysis to address this question, the present study analyzed medical journals extracted from databases such as Web of Science and Ulrich’s Periodicals Directory. The results indicate a causal relationship between JIFs and OA, and specifically that: (1) OA enhances JIFs; (2) countries that are less developed in science and technology are more likely to choose OA for publishing papers; (3) authors’ publication language and number of published papers, as well as publication release cycles all have significant effects on JIFs.

 

The effect of “open access” on journal impact factors: A causal analysis of medical journals – NASA/ADS

Abstract:  The Journal Impact Factor (JIF) has a significant influence on authors of research paper submissions. Whether open access (OA) is beneficial to JIFs and the dissemination of academic research results remains an under-examined issue. Using panel data analysis to address this question, the present study analyzed medical journals extracted from databases such as Web of Science and Ulrich’s Periodicals Directory. The results indicate a causal relationship between JIFs and OA, and specifically that: (1) OA enhances JIFs; (2) countries that are less developed in science and technology are more likely to choose OA for publishing papers; (3) authors’ publication language and number of published papers, as well as publication release cycles all have significant effects on JIFs.

 

Open access efforts begin to bloom: ESC Heart Failure gets full attention and first impact factor

Abstract:  In 2014, the Heart Failure Association (HFA) of the European Society of Cardiology (ESC) founded the first open access journal focusing on heart failure, called ESC Heart Failure (ESC?HF). In the first 5 years, in ESC?HF we published more than 450 articles. Through ESC?HF, the HFA gives room for heart failure research output from around the world. A transfer process from the European Journal of Heart Failure to ESC?HF has also been installed. As a consequence, in 2018 ESC?HF received 289 submissions, and published 148 items (acceptance rate 51%). The journal is listed in Scopus since 2014 and on the PubMed website since 2015. In 2019, we received our first impact factor from ISI Web of Knowledge / Thomson?Reuters, which is 3.407 for 2018. This report reviews which papers get best cited. Not surprisingly, many of the best cited papers are reviews and facts & numbers mini reviews, but original research is also well cited.

Open access efforts begin to bloom: ESC Heart Failure gets full attention and first impact factor – Anker – 2019 – ESC Heart Failure – Wiley Online Library

Abstract:  In 2014, the Heart Failure Association (HFA) of the European Society of Cardiology (ESC) founded the first open access journal focusing on heart failure, called ESC Heart Failure (ESC?HF). In the first 5 years, in ESC?HF we published more than 450 articles. Through ESC?HF, the HFA gives room for heart failure research output from around the world. A transfer process from the European Journal of Heart Failure to ESC?HF has also been installed. As a consequence, in 2018 ESC?HF received 289 submissions, and published 148 items (acceptance rate 51%). The journal is listed in Scopus since 2014 and on the PubMed website since 2015. In 2019, we received our first impact factor from ISI Web of Knowledge / Thomson?Reuters, which is 3.407 for 2018. This report reviews which papers get best cited. Not surprisingly, many of the best cited papers are reviews and facts & numbers mini reviews, but original research is also well cited.

Open and Shut?: The OA Interviews: K. VijayRaghavan, Principal Scientific Adviser, Government of India

“It is, however, clearly problematic that cOAlition S has remained an essentially European initiative. For this reason when, in February, the Indian Government’s Principal Scientific Adviser, Professor VijayRaghavan posted a series of tweets saying that India was joining cOAlition S the news was greeted with great excitement by cOAlition S members, as well as by Plan S supporters like the European Commissioner for Research, Science and Innovation Carlos Moedas.

 

The news was greeted with less enthusiasm back home in India, with concerns raised about the cost implications, the likely impact on small journals and publishers, and the way in which it would allow commercial publishers to continue to profit excessively from the research community – see, for instance, here, here and here.

 

Following Prof. VijayRaghavan’s tweets, however, radio silence set in, with no confirmation that India had formally joined, or any updates on the status of its plans. For this reason many ears pricked up last Friday when, during a lecture he gave at IISc Bangalore to mark Open Access Week, Prof. VijayRaghavan commented, “We are not committed to whatever Plan S does or does not do.” This sufficiently piqued the interest of Vasudevan Mukunth that he sought out Prof. VijayRaghavan and asked for clarification, which led to an interview in The Wire where it was confirmed that India no longer plans to join cOAlition S.

 

As I had been trying to interview Prof. VijayRaghavan for some months, I too was piqued by his comments and so took to Twitter to again invite him to answer the questions I had sent him in June. He agreed and below are his answers to an updated list of questions I emailed over to him….”