Which Academic Search Systems are Suitable for Systematic Reviews or Meta?Analyses? Evaluating Retrieval Qualities of Google Scholar, PubMed and 26 other Resources – Gusenbauer – – Research Synthesis Methods – Wiley Online Library

Abstract:  Rigorous evidence identification is essential for systematic reviews and meta?analyses (evidence syntheses), because the sample selection of relevant studies determines a review’s outcome, validity, and explanatory power. Yet, the search systems allowing access to this evidence provide varying levels of precision, recall, and reproducibility and also demand different levels of effort. To date, it remains unclear which search systems are most appropriate for evidence synthesis and why. Advice on which search engines and bibliographic databases to choose for systematic searches is limited and lacking systematic, empirical performance assessments.

This study investigates and compares the systematic search qualities of 28 widely used academic search systems, including Google Scholar, PubMed and Web of Science. A novel, query?based method tests how well users are able to interact and retrieve records with each system. The study is the first to show the extent to which search systems can effectively and efficiently perform (Boolean) searches with regards to precision, recall and reproducibility. We found substantial differences in the performance of search systems, meaning that their usability in systematic searches varies. Indeed, only half of the search systems analysed and only a few Open Access databases can be recommended for evidence syntheses without adding substantial caveats. Particularly, our findings demonstrate why Google Scholar is inappropriate as principal search system.

We call for database owners to recognise the requirements of evidence synthesis, and for academic journals to re?assess quality requirements for systematic reviews. Our findings aim to support researchers in conducting better searches for better evidence synthesis.

Two new kids on the block: How do Crossref and Dimensions compare with Google Scholar, Microsoft Academic, Scopus and the Web of Science?

Abstract:  In the last 3 years, several new (free) sources for academic publication and citation data have joined the now well-established Google Scholar, complementing the two traditional commercial data sources: Scopus and the Web of Science. The most important of these new data sources are Microsoft Academic (2016), Crossref (2017) and Dimensions (2018). Whereas Microsoft Academic has received some attention from the bibliometric commu-nity, there are as yet very few studies that have investigated the coverage of Crossref or Dimensions. To address this gap, this brief letter assesses Crossref and Dimensions cover-age in comparison to Google Scholar, Microsoft Academic, Scopus and the Web of Science through a detailed investigation of the full publication and citation record of a single academic, as well as six top journals in Business & Economics. Overall, this first small-scale study suggests that, when compared to Scopus and the Web of Science, Crossref and Dimensions have a similar or better coverage for both publications and citations, but a substantively lower coverage than Google Scholar and Microsoft Academic. If our find-ings can be confirmed by larger-scale studies, Crossref and Dimensions might serve as good alternatives to Scopus and the Web of Science for both literature reviews and citation analysis. However, Google Scholar and Microsoft Academic maintain their position as the most comprehensive free sources for publication and citation data

#DontLeaveItToGoogle: How Open Infrastructures Enable Continuous Innovation in the Research Workflow | Zenodo

Abstract:  Closed and proprietary infrastructures limit the accessibility of research, often putting paywalls in front of scientific knowledge. But they also severely limit reuse, preventing other tools from building on top of their software, data, and content. Using the example of Google Scholar, I will show how these characteristics of closed infrastructures impede innovation in the research workflow and create lock-in effects. I will also demonstrate how open infrastructures can help us move beyond this issue and create an ecosystem that is community-driven and community-owned. In this ecosystem, innovation thrives, as entry barriers are removed and systems can make use of each other’s components. Specific consideration will be given to open source services and non-profit frontends, as they are often overlooked by funders, but represent the way researchers engage with open science.

How Readers Discover Content in Scholarly Publications: Trends in reader behavior from 2005 to 2015

“Headline findings:

As a starting point for search, A&Is seem to be in a slight decline when looked at in aggregate across all regions and sectors, but remain the most important. Figure 4, p11 ? Academic researchers in high income countries now rate library discovery as highly as A&Is, and rate academic search engines as the most important discovery resource when searching for journal articles. Figure 11, p18 ? Library discovery services have made significant advances in importance in search for academic researchers, and for all roles in hard sciences in the academic sector. As an average across all subjects and sectors, however, they have not grown in importance in since 2012. Figure 4, p11; Figure 7, p14; Figure 11, p18 ? More than half of all journal content delivery appears to be from free incarnations of articles. There appears to be a clear PubMedCentral effect in the medical sector. Social media sites appear to be a significant source of free articles in lower income countries. Figure 37, p39 ? In academic STM in higher income countries, academic search engines are now more important than general search engines. Figure 4, p11 ? Table of Contents alerts have reduced in popularity in all measures across the survey. Figure 26, p31; Figure 35, p37 ? There appears to be an increased role for social media in discovery. Figure 9, p16 ? Online book discovery varies significantly by sector, with academics preferring library web pages marginally over general web search engines, the medical sector preferring A&I services and library web over search engines, but all other sectors preferring search engines over other forms of discovery. Figure 31, p34 ? Publisher web sites are becoming more popular as a search resource, although this is less true for people in wealthier countries. Figure 10, p17; Figure 18, p24 ? Google Scholar is used more than Google in the academic sector, but less than Google in all other sectors. Figure 22, p27 ? A perceived lack of awareness of Google Scholar in poorer nations appears to be leading to a reduced use of free incarnations of content in institutional repositories from these regions. Page 40 ? Readers in low income countries use their mobiles to access journals more than their counterparts in richer countries. However, access by phone still accounts for only about 10% of the use. Figure 42, p44 ? A&Is continue to be the most important search method in the medical sector. Figure 15, p21 ? The primary method of journals discovery is search, but even more so for online books. Figure 33, p35 ? App use for journal discovery is still low. Figure 45, p45 ? The most highly sought-after features of journal web sites are changing. Figure 49, p48 …”

Getting serious about open access discovery – Is open access getting too big to ignore? | Musings about librarianship

“With all the intense interest Unpaywall is getting (See coverage in academic sites like Nature, ScienceChronicle of Higher education, as well as more mainstream tech sites like TechcruchGimzo), you might be surprised to know that Unpaywall isn’t in fact the first tool that promises to help users unlock paywalls by finding free versions.

Predecessors like Open Access button (3K users), Lazy Scholar button (7k Users), Google Scholar button (1.2 million users) all existed before Unpaywall(70k users) and are arguably every bit as capable as Unpaywall and yet remained a niche service for years.”

Open Academic Search

“Open Academic Search (OAS) is a working group aiming to advance scientific research and discovery, promote technology that assists the scientific and academic communities, and make research available worldwide for the good of all humanity….Our core principles: [1] Collaboration drives innovation in academic search. [2] AI plays a unique role in surfacing and analyzing information in millions of research papers and academic journals. [3] Our core mission is advancing the pace of research and aiding breakthroughs in critical research areas….”

The dark side of Open Access in Google and Google Scholar: the case of Latin-American repositories – ScienceOpen

Abstract:  Since repositories are a key tool in making scholarly knowledge open access, determining their presence and impact on the Web is essential, particularly in Google (search engine par excellence) and Google Scholar (a tool increasingly used by researchers to search for academic information). The few studies conducted so far have been limited to very specific geographic areas (USA), which makes it necessary to find out what is happening in other regions that are not part of mainstream academia, and where repositories play a decisive role in the visibility of scholarly production. The main objective of this study is to ascertain the presence and visibility of Latin American repositories in Google and Google Scholar through the application of page count and visibility indicators. For a sample of 137 repositories, the results indicate that the indexing ratio is low in Google, and virtually nonexistent in Google Scholar; they also indicate a complete lack of correspondence between the repository records and the data produced by these two search tools. These results are mainly attributable to limitations arising from the use of description schemas that are incompatible with Google Scholar (repository design) and the reliability of web indicators (search engines). We conclude that neither Google nor Google Scholar accurately represent the actual size of open access content published by Latin American repositories; this may indicate a non-indexed, hidden side to open access, which could be limiting the dissemination and consumption of open access scholarly literature.