Collini on “Impact on humanities” in Times Literary Supplement


Commentary on:
Collini, S. (2009) Impact on humanities: Researchers must take a stand now or be judged and rewarded as salesmen. Times Literary Supplement. November 13 2009.

One can agree whole-heartedly with Professor Collini that much of the spirit and the letter of the RAE and the REF and their acronymous successors are wrong-headed and wasteful — while still holding that measures (“metrics”) of scholarly/scientific impact are not without some potential redeeming value, even in the Humanities. After all, even expert peer judgment, if expressed rather than merely silently mentalized, is measurable. (Bradley’s observation on the ineluctability of metaphysics applies just as aptly to metrics: “Show me someone who wishes to refute metaphysics and I’ll show you a metaphysician with a rival system.”)

The key is to gather as rich, diverse and comprehensive a spectrum of candidate metrics as possible, and then test and validate them jointly, discipline by discipline, against the existing criteria that each discipline already knows and trusts (such as expert peer judgment) so as to derive initial weights for those metrics that prove to be well enough correlated with the discipline’s trusted existing criteria to be useable for prediction on their own.

Prediction of what? Prediction of future “success” by whatever a discipline’s (or university’s or funder’s) criteria for success and value might be. There is room for putting a much greater weight on the kinds of writings that fellow-specialists within the discipline find useful, as Professor Collini has rightly singled out, rather than, say, success in promoting those writings to the general public. The general public may well derive more benefit indirectly, from the impact of specialised work on specialists, than from its direct impact on themselves. And of course industrial applications are an impact metric only for some disciplines, not others.

Ceterum censeo: A book-citation impact metric is long overdue, and would be an especially useful metric for the Humanities.

Harnad, S. (2001) Research access, impact and assessment. Times Higher Education Supplement 1487: p. 16.

Harnad, S., Carr, L., Brody, T. & Oppenheim, C. (2003) Mandated online RAE CVs Linked to University Eprint Archives: Improving the UK Research Assessment Exercise whilst making it cheaper and easier. Ariadne 35.

Brody, T., Carr, L., Harnad, S. and Swan, A. (2007) Time to Convert to Metrics. Research Fortnight pp. 17-18.

Harnad, S. (2008) Open Access Book-Impact and “Demotic” Metrics Open Access Archivangelism October 10, 2008.

Harnad, S. (2008) Validating Research Performance Metrics Against Peer Rankings. Ethics in Science and Environmental Politics 8 (11) doi:10.3354/esep00088 Special Issue on “The Use And Misuse Of Bibliometric Indices In Evaluating Scholarly Performance”

Harnad, S. (2009) Open Access Scientometrics and the UK Research Assessment Exercise. Scientometrics 79 (1)

The Elephant in the Room

Fred Friend, Honorary Director Scholarly Communication UCL, wrote in liblicense:

FF: “The phrase “the elephant in the room” was used by a librarian at a recent UK meeting to describe the big issues we were not allowed to discuss about how the current economic crisis is affecting scholarly communication. Representatives of all stakeholder groups present – including publishers – agreed that the economic crisis was hitting them badly, with cost-cutting happening across the board and hopes for growth put on hold. The curious feature of the conversation was that nobody present was able to discuss the one topic which could get us through the crisis and prevent the journals market collapsing, viz. the pricing structure for journal “big deals”. Pricing can only be discussed in one-to-one meetings between suppliers and purchasers. It would be easy to blame legislators for anti-trust legislation and the dominance of contract law, but the legal web within which publishing is entwined is of our own making – and I include the academic community in that statement.

“The importance of this failure to discuss structural and pricing issues is that the dominance of library budgets by “big deal” expenditure has the potential to bring the journal publishing industry to its knees in the same way as sub-prime mortgages did for the banking industry. It will only take a few cancellations of “big deals” by major institutions to make investors nervous about the future of companies heavily dependent upon such deals, and a domino effect could follow. We may be sure that there will be no government bail-out of the journal publishing industry. This scenario would not be good for any of the current stakeholders. The big journal publishing companies have failed to respond positively to the ICOLC initiative on the economic crisis, and the inability to discuss structural and pricing issues in a collaborative way is preventing solutions which have been of benefit in other sectors of the economy. For example, heavily-discounted pricing (by which I do not mean 1%) could ease the burden upon library budgets for one or two years until the overall economic situation improved. No publisher will want to be the first to discuss such solutions, but equallly no publisher will want to be the first to feel the effects of cancellations of its ‘big deals’.”

The elephant in the room is not the prospect of journal subscription cancellations, because as long as researchers need access to peer-reviewed journals, and as long as peer-reviewed journals are accessible only via subscriptions, subscriptions will remain viable, and institutions will just have to keep paying for whatever fraction they can afford of them.

The elephant in the room is Open Access (OA) self-archiving of journal articles by the “Slumbering Giant” — the universal provider of all the content of the planet’s 25,000 peer-reviewed journals: the planet’s 10,000 universities and research institutions.

As soon as the Slumbering Giant awakes to the fact that OA is fully within its reach — all it has to do is to mandate it — all the fuss about journal affordability, institutional serials crises, and publisher overpricing will fade, for researchers will have access to all refereed research, not just the fraction of it to which their institutions can afford to subscribe today.

And then, maybe, institutions will start canceling, their users’ needs no longer being inelastic, thanks to the OA mandates.

And then publishers will cut costs, lower prices, and eventually make a transition to OA publishing, recovering the costs of peer review from institutional publication fees, paid out of a fraction of institutions’ windfall subscription cancellation savings.

That’s the real elephant in the room, if you like. But as long as we persist in imagining instead that it’s something to do with journal pricing, “Big Deals,” and the need for pricing reform, we will not only fail to notice the elephant, we will fail to grasp its tail, which is fully within our reach. Instead, we will, like the drunk and the lamp-post, keep fumbling where the elephant isn’t (or, like the blind men and the elephant, fail to grasp what it is)!

Ganesh Loxodont

6. Joining things up

A national portal designed to raise the profile of Open Accessresearch in Ireland. The Irish-African Partnership for Research Capacity Building Digital repository for the participating Irish and African universities.
Niamh Brennan, Programme Manager, Research Information Systems & Services, Trinity College Library Dublin, Ireland

OA McMemberships, Dismemberment and MC Escher

Gold OA institutional “membership” is incoherent and does not scale. It only gives the illusion of making sense if you think of it locally, and myopically. Annual institutional subscriptions to journals containing the annual outgoing refereed research of all other institutions do not morph into annual institutional memberships for publishing each institution’s own outgoing refereed research. There are 25,000 journals and 10,000 institutions! Is every single institution to commit and contract in advance to pay for its authors’ (potential) fraction of annual submissions to every single journal? Is that a “membership” or a distributed dismemberment? And is every journal to commit and contract in advance to accept every institution’s annual fraction of submissions? (Is that peer review?) This is a global oligopolistic illusion that would fit publishers just about as well as it would fit McDonalds, except there are at least 25,000 different journals to “join”, and institutions each have thousands of author-consumers with diverse dietary needs, varying day to day and year to year.

Part of the illusion of coherence comes from thinking in terms of journal-fleet publishers instead of individual journal article submissions. But this is merely another variant of the “Big Deal” strategy that has done nothing to solve either the accessibility or the affordability problem. The reality is that (Gold) OA publishing is premature today, except as a proof of principle. What is needed first is for universal (Green) OA self-archiving mandates to be adopted by institutions and funders. That will provide universal (Green) OA, which may eventually generate cancellation pressure that will induce journals to cut obsolete costs and products/services by downsizing to just providing peer review, paid for by individual institutions on an individual outgoing article basis out of a fraction of their annual windfall savings from their institutional subscription cancellations. To buy into “memberships” with fleet publishers now, pre-emptively, and at current prices, while the money is still tied up in subscriptions (which cannot, of course, be cancelled in advance, before OA) is both penny- and pound-foolish — and downright absurd if a “member” institution has not even first mandated Green OA self-archiving for all of its own refereed research output…

Stevan Harnad
American Scientist Open Access Forum