‘Undue reliance’ on journal impact factor in academic evaluation | Times Higher Education (THE)

At least one in three research-intensive universities in North America examined by a study leaned on the journal impact factor of periodicals that academics had published in when making decisions on promotion and tenure, but the true proportion may be much higher.

The study, believed to be the first to examine the use of the journal impact factor in academic performance reviews, warns that there is an “undue reliance” on the controversial metric….

Among the documents from 57 research-intensive institutions considered by the study, 23 (40 per cent) referred to journal impact factors, with 19 of these mentions (83 per cent of the subtotal) being supportive. Only three of the mentions expressed caution about use of journal impact factors.

Of the documents that did refer to journal impact factors, 14 associated the metric with research quality, while eight tied it to impact and a further five referred to prestige or reputation.

The overall results, including large numbers of universities that offer few doctoral degrees, found that 23 per cent of review, promotion and tenure policies mentioned the journal impact factor, with 87 per cent of these mentions being supportive….”

Ithaka S+R US Faculty Survey 2018 | Ithaka S+R

Faculty have displayed a notable increase in interest for an open access publication system since the last survey cycle. Approximately 64% of respondents in 2018 indicated they would be happy to see the traditional subscription-based publication model replaced entirely by an open access system compared to 57% in 2015.

Younger faculty are more interested than their older colleagues in replacing the traditional subscription-based system with an open access one (See Figure 32). This is perhaps surprising given that older faculty place more importance on the characteristics of open access when deciding in which journals to publish (see Figure 31). Older faculty are also more likely to understand the criteria used in tenure and promotion evaluations, and are less likely than younger faculty to shape their research outputs and publication choices to match the criteria for success in tenure and promotion (see Figure 32). This suggests that older faculty, who are often more established, published, and/or tenured, may make their publications and findings open because the traditional scholarly incentives are not as relevant for them….

Impact factors are still widely used in academic evaluations

Almost half of research-intensive universities consider journal impact factors when deciding whom to promote, a survey of North American institutions has found.

About 40% of institutes with a strong focus on research mention impact factors in documents used in the review, promotion and tenure process, according to the analysis, which examined more than 800 documents across 129 institutions in the United States and Canada.

The data imply that many universities are evaluating the performance of their staff using a metric that has been widely criticized as a crude and misleading proxy for the quality of scientists’ work….

Less than one-quarter of the institutions mentioned impact factor or a closely related term such as “high impact journal” in their documents. But this proportion rose to 40% for the 57 research-intensive universities included in the survey. By contrast, just 18% of universities that focused on master’s degrees mentioned journal impact factors (see ‘High impact’).

In more than 80% of the mentions at research-heavy universities, the language in the documents encouraged the use of the impact factor in academic evaluations. Only 13% of mentions at these institutions came with any cautionary words about the metric. The language also tended to imply that high impact factors were associated with better research: 61% of the mentions portrayed the impact factor as a measure of the quality of research, for example, and 35% stated that it reflected the impact, importance or significance of the work….”

Universities should be working for the greater good | Times Higher Education (THE)

What might happen if the provost of a highly visible research university that had recently reconfirmed its public-facing mission gathered the entire campus together – deans, department chairs and faculty – in rethinking the university’s promotion and tenure standards from top to bottom? What might become possible if that provost were to say that our definitions of “excellence” in research, teaching and service must have that public-facing mission at their heart? What might be possible if that public mission really became Job One?

The provost paused. Then he gave his answer: “Any institution that did that would immediately lose competitiveness within its cohort.” …

The pursuit of prestige is not the problem in and of itself, and excellence is, of course, something to strive for. In fact, friendly competition can push us all to do better. But excellence and prestige and the competitiveness that fuels their pursuit are too often based in marketing – indeed, in the logic of the market – rather than in the actual purposes of higher education. It’s a diversion from the on-the-ground work of producing and sharing knowledge that can result in misplaced investments and misaligned priorities….”

 

Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations [PeerJ Preprints]

Abstract:  The Journal Impact Factor (JIF) was originally designed to aid libraries in deciding which journals to index and purchase for their collections. Over the past few decades, however, it has become a relied upon metric used to evaluate research articles based on journal rank. Surveyed faculty often report feeling pressure to publish in journals with high JIFs and mention reliance on the JIF as one problem with current academic evaluation systems. While faculty reports are useful, information is lacking on how often and in what ways the JIF is currently used for review, promotion, and tenure (RPT). We therefore collected and analyzed RPT documents from a representative sample of 129 universities from the United States and Canada and 381 of their academic units. We found that 40% of doctoral, research-intensive (R-type) institutions and 18% of master’s, or comprehensive (M-type) institutions explicitly mentioned the JIF, or closely related terms, in their RPT documents. Undergraduate, or baccalaureate (B-type) institutions did not mention it at all. A detailed reading of these documents suggests that institutions may also be using a variety of terms to indirectly refer to the JIF. Our qualitative analysis shows that 87% of the institutions that mentioned the JIF supported the metric’s use in at least one of their RPT documents, while 13% of institutions expressed caution about the JIF’s use in evaluations. None of the RPT documents we analyzed heavily criticized the JIF or prohibited its use in evaluations. Of the institutions that mentioned the JIF, 63% associated it with quality, 40% with impact, importance, or significance, and 20% with prestige, reputation, or status. In sum, our results show that the use of the JIF is encouraged in RPT evaluations, especially at research-intensive universities, and indicates there is work to be done to improve evaluation processes to avoid the potential misuse of metrics like the JIF.

Elsevier chief: end citation fixation to halt ‘salami publishing’ | Times Higher Education (THE)

Mr Chi said that when he joined Elsevier 14 years ago, he “saw that we were publishing a lot because that was a way to make more money, but that wasn’t really serving the long-term benefit of our company or the community of researchers”. Acting on that insight, he said, the publishing giant decided to publish fewer papers but of higher quality.

As a consequence, Elsevier “lost several percentage points of market share in those 14 years” but “gained about 25 per cent in the FWCI [field-weighted citation index], which means that we really raised the quality of the papers we publish”. 

While he stressed that he was “not at all against the goal of the open access model”, Mr Chi said that Elsevier’s emphasis on high-grade work in effect opened a publishing niche and “left others to fill the vacuum”, which they did by publishing “without the quality control” and sometimes “without peer review”. …”

Worried About the Future of the Monograph? So Are Publishers – The Chronicle of Higher Education

From your perspective as the AUP’s new president, what are the most important issues facing scholarly publishers?

Crewe: Our biggest challenge remains the low sales of scholarly monographs, such as revised dissertations or scholarly books with a narrow focus in a small field. Libraries share copies, and individuals don’t purchase the new books in their fields as they did 20 years ago.

We want to publish these books. They are the building blocks of our own reputation and they are often groundbreaking, field-changing works. We’re looking for publishing grants to support them, and we try each season to publish enough profitable books to cover the losses on monographs.

But today’s model isn’t sustainable. There are a number of experiments under way to figure out how to publish specialized monographs in a freely available open-access format….”

Open-Access Is Going Mainstream. Here’s Why That Could Transform Academic Life. – The Chronicle of Higher Education

“That may soon change. Smaller-scale efforts are mixing with top-down decisions — through universities’ subscription negotiations and a major European plan that mandates open-access publication for certain research — to put unusual pressure on publishers.

Don’t think these battles are confined to the library or an individual discipline. The changes have the potential to alter nearly everything about how research is disseminated — and therefore how departments spend money, researchers collaborate, and faculty careers advance….”

Future of scholarly publishing and scholarly communication – Publications Office of the EU

“The report proposes a vision for the future of scholarly communication; it examines the current system -with its strengths and weaknesses- and its main actors. It considers the roles of researchers, research institutions, funders and policymakers, publishers and other service providers, as well as citizens and puts forward recommendations addressed to each of them. The report places researchers and their needs at the centre of the scholarly communication of the future, and considers knowledge and understanding created by researchers as public goods. Current developments, enabled primarily by technology, have resulted into a broadening of types of actors involved in scholarly communication and in some cases the disaggregation of the traditional roles in the system. The report views research evaluation as a keystone for scholarly communication, affecting all actors. Researchers, communities and all organisations, in particular funders, have the possibility of improving the current scholarly communication and publishing system: they should start by bringing changes to the research evaluation system. Collaboration between actors is essential for positive change and to enable innovation in the scholarly communication and publishing system in the future….”