Online Platforms for Recruiting and Motivating Reviewers

Authors and publishers have easily understandable motivations for participating in scholarly publishing, but there is less clear motivation for reviewers. This post highlights the need of recognizing and rewarding reviewers and describes how online platforms can ease achieving this objective at the time of being a source for recruiting reviewers and recording review activity. A description and a comparison of the main online platforms available today are also provided.

The academic publishing process is driven by four main actors: authors, editors, publishers and reviewers, each of whom play a vital role in ensuring that high standards are maintained throughout the process of preparing the article, reviewing it and finally publishing it. Each main actor needs to have some motivation that drives participation and the quality of their contribution to the publishing process. I would like to summarize what I think are the main motivations of each party in the review process. Authors are driven by their wish to make public the results of their investigations. Besides that, the production of high quality scientific content is a highly valued merit in academia and research. Researchers whose curricular vitae boast of a large list of high-quality publications are well respected and have easier access to funding.
When it comes to editors, becoming a member of editorial board of scientific journals is in itself considered to be a merit. Editors normally serve in an “altruistic” mode, without expecting financial reward. They view being an editor as a means by which they can give back to the scientific and academic community. However, some editors are perhaps not as altruistic as one may think since they also gain recognition from the role which enhances their reputations and therefore access to funding. In addition, it is noteworthy that some publishers do provide some sort of compensation to editors for their work, which can be an additional motivation.
Scientific publishers are mainly based on two models of publication: 1. The traditional model, in which access to the full text of the articles is only accessible to subscribers (individual or institutional) 2. Open access model, in which publishers charge authors a fee for publishing articles with the full text available to all readers. In one way or other, major publishers manage to generate large chunks of revenue from the publishing process. The scientific publishing industry alone generates billions of dollars every year (1-4). Besides this, there is also a large group of non-profit and association/institutional publishers who make very little (if any) financial gain from their journals, but publish them as part of their mission to serve members and academia. Thus, the motivation of this last type of non-profit journal is radically different of that of publishers working as traditional for-profit companies.
While the motivation for three of the four actors in the publishing process can be clearly identified, the reason why reviewers participate in the publishing process is not so clear. There is no “material” reward for reviewers. Rather, it is the scientific altruism or commitment to the scientific model that motivates them to work. Reviewers are encouraged by the belief that they play an important role in ensuring that good quality research work reaches the community. The fact that reviewers themselves are also authors makes them more aware of the importance of good reviewers. In recent decades the number of scientific journals and the number of published articles has multiplied with a growth rate of approximately 3%-10% per year depending on the research area  (5-8), resulting in a true “explosion” of manuscripts that are submitted to publishers. As journals receive more and more manuscripts and the number of journals continues to grow, reviewers get saturated with multiple requests and invitations.  Thus, it is easy to understand “reviewer fatigue”, although many other factors may influence the reviewer’s decision to decline invitations to review manuscripts   (9). As a consequence editors often cannot find appropriate reviewers for manuscripts and this may result in delayed times for the various phases of the review process, and authors often have to wait months until their manuscripts get reviewed.
Getting more reviewers and making them more committed with providing good review reports on time is the main reason why it is necessary to increase the motivation of the reviewers. And indeed it seems fair to reward authors for their work in a sector that generates significant benefits. Several voices insist on this need again and again worldwide (10-15). Some journals/publishers are experimenting with direct payment of reviewers, although this is an exception. Anyway several arguments can be made against direct monetary compensation, in particular because paying reviewers would break the independence between editors/publishers and reviewers, which is one of the pillars of the academic publishing process. Most publishers acknowledge reviewers in front-matter summary pages or lists of reviewers or in letters upon request. Some others, such as Frontiers, make public the names of reviewers (and the name of the editor in charge) of all published articles including the names of the reviewers in a footnote in every published article. Others, such as Elsevier, are launching their own recognition platforms providing their reviewers with a personalized profile page where their reviewing history is documented and where they can download certificates. Authors and editors can also evaluate the quality of reviews done, providing feedback that may result in better quality of the review process. Nature, for example, recognizes reviewers with payment in kind, where reviewers receive free journal access, tools and services or vouchers for research supplies (16).  
In recent years, independent communities have developed online platforms offering review services for the scientific community. These platforms establish that it is possible to create an independent system where reviewers get recognition and reward for the efforts they put into ensuring that quality research reaches the scientific community. One of the main features of these platforms is that they are “third party companies” independent of publishers. This way, biases are completely prevented since editors and publishers are unable to influence reviewers, even when they may have a role in the workflow, since these platforms are designed to prevent direct communication among the different actors.
Basically, what these platforms do is provide authors and publishers with appropriate reviews and also provide reviewers with an extra motivation making them more willing to review manuscripts and complete the task in shorter periods (10, 11). They provide rewards to reviewers using two major strategies: 1. Credit through certificates or other elements that the reviewer can add to his curriculum vitae and 2. Other benefits such as monetary reward or rights to have their own manuscripts reviewed.
In this update, we report the global features of five of these platforms at the time of comparing them: Rubriq, Peereviewers, Publons.Peerage of Science and Academic Karma (Table 1). 

Rubriq
Peereviewers
Publons
Peerage of Science
Academic Karma
Service/s
Clients choose: review of contents + statistics, or review of contents + suggestion of suitable journals
Database of reviewers
Record of reviewers, journals and reviews
Reviews and publishing offers
Exchange of services
Review protocol
Closed. All manuscript go under the same protocol (Scorecard)
Open. Clients can customize the protocol of review
Open (Peerage Essay)
Open. Clients can customize the protocol of review
Fee (valid in  2015)
Several options depending on the services, from $500 to $650 (3 reviewers included)
$100 per reviewer
Type of acknowledgment to reviewers
Monetary (100$)
Monetary (50$), Certificate
Online record
Online record, ability to submit own articles for review
Online record, ability to submit own articles for review
Table 1. Comparison between third-party platforms offering reviewer services

To start with we would like to compare Rubriq (17) and Peereviewers (18). Both perform similarly but there are also some points distinguishing them (Table 1). In both cases, the reviewer must register on the platforms (restricted to academics and researchers with a given expertise) and declare their expert profile, so that they can be invited as reviewers for manuscripts that match their profile. Reviewers who are selected to review receive an email which contains a summary of the manuscript and instructions on how to complete the process. If the reviewer agrees, he/she will get access to the full text and the review form. When the review is finished a report is sent to the client and the reviewer is rewarded. The identity of the reviewer is also “anonymised” to the clients.
Another platform offering rewards to reviewers is Publons (19). Publons has a different objective: they do not offer any service to authors or publishers, but keep a record of reviewers, journals and reviews. They have a list of journals and create an account for each reviewer. A list containing all reviews conducted by a reviewer is listed in the reviewer’s account after being verified, next to the title of the journal to which each review belongs. Reviewers can claim the reviews they made in several ways, including online forms or by email. These data generate some statistics that place each reviewer in the corresponding percentile activity compared with that of all registered reviewers. The profile of each reviewer is public, so that reviewers can use this website to provide evidence of their activity.
Peerage of Science offers a tripartite where authors, reviewers and editors have a role (20) (Table 1). Authors submit manuscripts to Peerage of Science before submitting to any journal. Once submitted, any qualified peer-reviewer can choose to review the manuscript. The peer review process is available concurrently to all editors, with automated event tracking. If authors have received publishing offers from editors they may choose to accept one of these offers, or accept none and use their review in non-participating journals. A positive aspect of Peerage of Science is that peer reviewers are themselves peer reviewed. Reviewers are notified that they can evaluate the reviews sent by other reviewers. This extra twist contributes to increasing the quality of peer review. From the reviewer’s point of view, Peerage of Science offers credit for curricular purposes only as an externally verifiable measure of the reviewers’ expertise in their scientific fields.
An innovative approach comes from Academic Karma (21). Academic Karma is both the name of a currency and a platform for peer review. Instead of exchanging money, authors and reviewers exchange karma: reviewers earn 50 karma per reviewed manuscript and authors of the manuscript collectively spend 50 karma per reviewer (Table 1). Then reviewers may use their Karma for paying reviewers when authoring manuscripts. Editors are also involved since they receive the reviewer’s report simultaneously to authors.
An important point is how reviewers’ identities and their expertise are verified and how attribution of merits can be recorded and tracked. The Working Group on Peer Review Service (created to develop a data model and citation standard for peer review activity that can be used to support both existing and new review models) stresses the need for standardized citation structures for reviews which can enable the inclusion of peer review activity in personal recognition and evaluation, as well the ability to refer to reviews as part of the scholarly literature (6). In this regard, all platforms described here are using or are starting to use ORCID identifiers for both authors and reviewers, and DOIs as identification for published reviews (22). ORCID itself is also offering the option of adding reviews to ORCID profiles. Researchers with a profile in these networks can link this to their ORCID iD so that the reviews they have recorded on the platform are added to their ORCID page (23). In turn, these identificators will ease future reaearch on peer review and will probaly allow us to measure the impact of these platforms in the academic publishing process.
In conclusion, motivating and rewarding reviewers is a need that can be addressed both by publishers and third party organizations. Online platforms are good tools for giving credit to reviewers and to convey monetary reward, at the same time offering a way of recording review activity.

References and Notes
1.The Wellcome Trust (2003) Economic analysis of scientific research publishing: A report commissioned by the Wellcome Trust, revised ed. Available: http://www.wellcome.ac.uk/stellent/groups/corporatesite/@policy_communications/documents/web_document/wtd003182.pdf. Accessed 10th July 2015.
2. Costs and business models in scientific research publishing A report commissioned by the Wellcome Trust. http://www.wellcome.ac.uk/stellent/groups/corporatesite/@policy_communications/documents/web_document/wtd003184.pdf
3. The National Academies (US) Committee on Electronic Scientific, Technical, and Medical Journal Publishing. Electronic Scientific, Technical, and Medical Journal Publishing and Its Implications: Report of a Symposium. http://www.ncbi.nlm.nih.gov/books/NBK215820/
4. Ware, Mark and Mabe, Michael (2015)  An overview of scientific and scholarly journal publishing. International Association of Scientific, Technical and Medical Publishers, 2015. http://www.stm-assoc.org/2015_02_20_STM_Report_2015.pdf Accessed 20th October 2015.
5. Walker R, Rocha da Silva P. (2015) Emerging trends in peer review—a survey. Frontiers in Neuroscience 9:169
6. Paglione LD, Lawrence RN. (2015) Data exchange standards to support and acknowledge peer-review activity. Learned Publishing, 28 (4):309-316(8)
7.Van Noorden, R. (2014) Global scientific output doubles every nine years. Nature.com [Internet], NewsBlog, 7 May 2014. Available from: http:// blogs.nature.com/news/2014/05/global-scientific-output-doublesevery-nine-years.html
8. The Wellcome Trust (2015) Scholarly Communication and Peer Review: The Current Landscape and Future Trends. http://www.wellcome.ac.uk/stellent/groups/corporatesite/%40policy_communications/documents/web_document/wtp059003.pdf Accessed 12 November 2015.
9. Marijke Breuning, Jeremy Backstrom, Jeremy Brannon, Benjamin Isaak Gross, Michael Widmeie  Reviewer Fatigue? (2015) Why Scholars Decline to Review their Peers’ Work PS: Political Science & Politics 48(4):595-600. http://dx.doi.org/10.1017/S1049096515000827
10.Björk B; Hedlund T.(2015)  Emerging new methods of peer review in scholarly journals. Learned Publishing 28(2): 85-91
11. Thomson Reuters (2010) Increasing the Quality and Timeliness of Scholarly Peer Review. A report for Scholarly Publishers..http://scholarone.com/media/pdf/peerreviewwhitepaper.pdf
12. Taylor & Francis (2015) Peer review in 2015: A global view. http://authorservices.taylorandfrancis.com/peer-review-in-2015/Accessed 20th October 2015
13. Alice Meadows (2015, January 7th) Recognition for peer review and editing in Australia – and beyond? Blog post in Exchanges http://exchanges.wiley.com/blog/2015/01/07/recognition-for-peer-review-and-editing-in-australia-and-beyond/Accessed 20th October 2015.
14. Andrew Trounson. Journals should credit editors, says ARC. Post in The Australian http://www.theaustralian.com.au/higher-education/journals-should-credit-editors-says-arc/story-e6frgcjx-1227201178857Accessed 20th October 2015.
15. Alberts, P., Hanson, B., and Kelner, K.L. 2008. Reviewing peer review. Science, 321 (5885): 15. http://dx.doi.org/10.1126/science.1162115.
16. Review rewards. Nature [Internet], 514(7522): 274–274. http:// dx.doi.org/10.1038/514274a
17.http://www.rubriq.com/
18.http://www.peereviewers.com/
19.http://www.publons.com
20.https://www.peerageofscience.org
21.http://academickarma.org/
22.Gasparyan AY, Akazhanov NA, Voronov AA, Kitas GD. Systematic and open identification of researchers and authors: focus on open researcher and contributor ID. J Korean Med Sci. 2014 Nov;29(11):1453-6. doi: 10.3346/jkms.2014.29.11.1453
   

Open Access — What Do Authors Really Want?

There’s no doubt that open access (OA) is becoming more and more popular with authors. A recent article by Mikael Laakso and Bo-Christer Björk, published in BioMedCentral Medicine,found that in 2011 around 340,000 articles were published in just over 6,700 OA journals (defined as those included in the Directory of Open Access Journals) – approximately 17% of the total papers published in 2011. Of these, about half (166,700) were published as gold OA articles — journals requiring the payment of an article publication charge (APC).
So what do authors themselves think about OA? Does it affect where they choose to publish? What are their reasons for publishing – or not publishing – in an OA journal? The results of a recent Wiley survey provide some interesting answers.
In May 2012, we contacted more than 100,000 of our journal authors asking for feedback on their attitude to OA publishing (defined as where the author, their institution, or funding body pays a fee to ensure that the article is made free for all to read, download, and share online). Of the 10,600 who responded, about one-third have already published at least one OA article, while nearly 80% believe that OA is growing in their field (note, proportionately more researchers from the biological sciences responded than from any other field).
Inevitably, there are significant variations by subject discipline. Over half of all responding authors who have already published in an OA journal work in health-related or biological sciences; conversely, only 12% work in the social sciences. The top three barriers to OA publishing are:
  1. A dearth of high-profile titles (especially true in areas such as social sciences)
  2.    Lack of funding
  3. Concerns over quality.
Professional research experience also appears to impact an author’s decision to publish OA – researchers with five or more years experience are significantly more likely to have published an OA article than those with less (83% versus 17%). Geographical location may also play a part, with half of all OA authors coming from the US, the UK, China, Germany, or Australia – with the exception of China, all countries where there has been pro-OA lobbying for some time.
OA is not currently a significant factor in where authors choose to publish – in fact it doesn’t even make the Top 10. Instead, authors cite factors such as subject area/scope of the journal, impact factor (IF), society affiliation, international authorship/readership, and production quality (e.g., editing, quality of figures, etc. as their main influences.
In response to the question, “I would publish in an OA journal if . . .,” respondents cited quality and profile issues as their main requirements (IF, rigorous peer review process, high production values, well-respected editorial board, well-regarded by peers), while value-added services such as speed of publication and article-level metrics are currently considered much less important. Conversely, the main reasons for not publishing in an OA journal are around concerns about profile, quality, and lack of funding, which is cited by 44% as a factor.
Funding seems to be a key issue, with only 18% of authors receiving full funding for article publication.  Additionally, young researchers are nearly twice as likely not to have funding as their older colleagues; somewhat counter-intuitively, however, they don’t see this as any more of a barrier to publishing OA.  Presumably OA will rapidly move up the ranks as a factor in where researchers choose to publish as funder mandates start to kick in – the introduction of the RCUK mandate on April 1, 2013, and corresponding increase in APC funding for articles arising from UK funded research, should be a good indicator.
Overall, the results are encouraging for the future of gold OA publishing. A continuing commitment on the part of publishers and societies to publish high quality, high IF journals through a strong and sustainable peer review process looks likely to attract increasing numbers of authors to publish in OA journals. And there are also opportunities to launch new high-quality OA journals in disciplines where these don’t currently exist – as long as the funding is available to support them.
But there’s the rub. If gold OA is to succeed as a means of providing immediate public access to research articles, then authors across all disciplines need better access to APC funding. While the UK government has already committed to finding some additional funds for APCs following publication of the Finch Group report, it is still not clear how much will be available. And in Brussels, there seems to be little likelihood of much additional funding for gold OA in the European Commission’s Horizon 2020 budget. Let’s hope that, in order to achieve the “significant social and economic benefits” of making research outputs freely available that the RCUK hopes for, they and other funding bodies will be prepared to put their money where their mouth is.
Published at The Scholarly Kitchen

Equity for Open-Access Journal Publishing

Scholars write articles to be read—the more access to their articles the better—so one might think that the openaccess approach to publishing, in which articles are freely available online to all without interposition of an access fee, would be an attractive competitor to traditional subscription-based journal publishing.
The new US administration could implement such a system through simple FRPAA-like legislation requiring funding agencies to commit to this openaccess compact in a cost-neutral manner. Perhaps reimbursement would be limited to authors at universities and research institutions that themselves commit to a similar compact. As funding agencies and universities take on this commitment, we might transition to an efficient, sustainable journal publishing system in which publishers choose freely among business models on an equal footing, to the benefit of all (Full text).