Perception of the importance of chemistry research papers and comparison to citation rates

Abstract:  Chemistry researchers are frequently evaluated on the perceived significance of their work with the citation count as the most commonly-used metric for gauging this property. Recent studies have called for a broader evaluation of significance that includes more nuanced bibliometrics as well as altmetrics to more completely evaluate scientific research. To better understand the relationship between metrics and peer judgements of significance in chemistry, we have conducted a survey of chemists to investigate their perceptions of previously published research. Focusing on a specific issue of the Journal of the American Chemical Society published in 2003, respondents were asked to select which articles they thought best matched importance and significance given several contexts: highest number of citations, most significant (subjectively defined), most likely to share among chemists, and most likely to share with a broader audience. The answers to the survey can be summed up in several observations. The ability of respondents to predict the citation counts of established research is markedly lower than the ability of those counts to be predicted by the h-index of the corresponding author of each article. This observation is conserved even when only considering responses from chemists whose expertise falls within the subdiscipline that best describes the work performed in an article. Respondents view both cited papers and significant papers differently than papers that should be shared with chemists. We conclude from our results that peer judgements of importance and significance differ from metrics-based measurements, and that chemists should work with bibliometricians to develop metrics that better capture the nuance of opinions on the importance of a given piece of research.

The academic papers researchers regard as significant are not those that are highly cited

“For many years, academia has relied on citation count as the main way to measure the impact or importance of research, informing metrics such as the Impact Factor and the h-index. But how well do these metrics actually align with researchers’ subjective evaluation of impact and significance? Rachel Borchardt and Matthew R. Hartings report on a study that compares researchers’ perceptions of significance, importance, and what is highly cited with actual citation data. The results reveal a strikingly large discrepancy between perceptions of impact and the metric we currently use to measure it.”

[Wellcome Trust consultation on revising its OA policy]

“Wellcome is conducting a review of its open access (OA) policy. The aim of this review is to ensure that knowledge and discoveries which arise from publications stemming from our funding are shared and used to maximise their benefit to health. This open consultation forms part of the review process and responses will be used by our internal working group to inform the outcome….”

News & Views: Trendspotting in OA Spending – Delta Think

“The findings in the report are not surprising: they match closely with Delta Think’s numbers from October, and with those in similar reports. Open access is established; it now covers significant minority share of output, but its growth is slowing:

  • Globally, publishers offer OA options mainly through hybrid journals: 72% of journals are hybrid, 19% fully OA, and 9% of journals are subscription only.
  • Uptake, in terms of articles suggests that 19% of all articles published are available immediately on publication as OA, split between 15% in fully OA journals and just under 4% in hybrid journals.
  • The report explores delayed OA options, giving a read on Green OA, with an uptake of just under 5% in the year of publication.
  • The 2017 update reduces its OA estimates slightly compared with its 2015 version. The results are summarized in the table below. The variations speak to the challenges in gathering data, and the necessity to keep refining models over time….”
  • APCs now form a significant additional expense. For the payments that universities make to the seven largest publishers, the ratio between subscriptions and APCs is 5:1. This equates to a 17% share of revenue compared to a 31% share of output. As we have discussed in our previous market analyses, on average the revenue generated by OA is proportionately less than its share of output.
  • More than half the expenditure on APCs in 2016 went to the three major publishing groups, Elsevier, Springer Nature, and Wiley, with a particularly sharp rise for Elsevier since 2014.
  • The report confirms the well-known finding that APCs for hybrid journals are more expensive that those in fully OA journals: 28% higher on average in 2016.
  • But, this gap may be closing: hybrid prices paid rose by 14% in the three years from 2013 (to £2,095 on average), but by 33% for fully-OA journals (to £1,640) in the same period. Delta Think’s market models suggest that, whilst hybrid prices are higher, they also bear higher discount levels, so it would appear that the gap between fully OA and hybrid prices is closing….
  • The number of APCs paid by a sample of 10 UK universities rose more than fivefold.
  • The average cost of an APC rose by 16% (as compared with a rise of 5% in the consumer price index; the CPI.)
  • Spending on subscriptions for the report’s sample rose by 20%.
  • Nuances within hybrid spending show the same quadrupling of APCs, with combined APC & sub spending up by one third…in other words, APC spending is eating share. The ratio between subscription and hybrid APC spending has fallen to 6:1 in 2016 from 19:1 3 years previously….”

Research data services for institutions | Open research | Springer Nature

“A small survey of global library staff reveals that respondents view open access as the future of academic and scientific publishing, and many are not satisfied with the current speed of the transition….

The majority of respondents thought that there would come a time when all future scholarly articles will be published open access, with two thirds believing this could happen over the next 10 years….”

Open Data: The Researcher Perspective

“The Open Data report is a result of a year-long, co-conducted study between Elsevier and the Centre for Science and Technology Studies (CWTS), part of Leiden University, the Netherlands.  The study is based on a complementary methods approach consisting of a quantitative analysis of bibliometric and publication data, a global survey of 1,200 researchers and three case studies including in-depth interviews with key individuals involved in data collection, analysis and deposition in the fields of soil science, human genetics and digital humanities….”

73 percent of academics say access to research data helps them in their work; 34 percent do not publish their data

“Combining results from bibliometric analyses, a global sample of researcher opinions and case-study interviews, a new report reveals that although the benefits of open research data are well known, in practice, confusion remains within the researcher community around when and how to share research data….”

BOAI 15 Survey Report

“The 15th anniversary of the Budapest Open Access Initiative provided an excellent opportunity to take stock of global progress toward open access and to gauge the main obstacles still remaining to the widespread adoption of open access policies and practices. As part of this process, feedback was solicited through an open survey that was disseminated online, and that received responses from individuals in 60 countries around the world.

Markers of progress are clear. The lack of understanding of the concept of open access and a myriad of misconceptions that were pervasive at the time of the BOAI’s original convening have receded, as open access has become a widely accepted fact of life

in research and scholarship. These have been supplanted by concerns that are more operational and nuanced in nature, essentially moving from debates about the “what and why” of open access to the “how“—how to best get it done.

The survey showed two clear primary challenges. First and foremost, respondents noted the lack of meaningful incentives and rewards for scholars and researchers to openly share their work. This challenge resonated at both the global level (56% of respondents in Figure 1) and the local level (29.5% of respondents in Table 1). This was followed by concern over a lack of funds to pay for APCs or other open access-related costs (36% of respondents in Figure 1; 28.3% of respondents in Table 1).

The results of the survey indicate the transition from establishing open access as a concept—which the BOAI did for the first time in 2002—to making open the default for research and scholarship. These two key challenges point to areas where concerted effort needs to be focused to continue making progress towards open access. Strategies to align incentives and rewards for scholars to share their work openly and the need to construct affordable, sustainable, and equitable business models to support open access publishing must be embraced as primary working priorities by the open access community….”