Citation analysis with microsoft academic

Authors: Hug SE, Ochsner M & Brandle MP

Comment: This article compares the citation analyses between Microsoft Academic (MA) and Scopus. This was compared via the output of three selected researchers. The results showed uniformity across MA and Scopus. Some limitations to MA were also pointed out.

Abstract: We explore if and how Microsoft Academic (MA) could be used for bibliometric analyses. First, we examine the Academic Knowledge API (AK API), an interface to access MA data, and compare it to Google Scholar (GS). Second, we perform a comparative citation analysis of researchers by normalizing data from MA and Scopus. We find that MA offers structured and rich metadata, which facilitates data retrieval, handling and processing. In addition, the AK API allows retrieving frequency distributions of citations. We consider these features to be a major advantage of MA over GS. However, we identify four main limitations regarding the available metadata. First, MA does not provide the document type of a publication. Second, the “fields of study” are dynamic, too specific and field hierarchies are incoherent. Third, some publications are assigned to incorrect years. Fourth, the metadata of some publications did not include all authors. Nevertheless, we show that an average-based indicator (i.e. the journal normalized citation score; JNCS) as well as a distribution-based indicator (i.e. percentile rank classes; PR classes) can be calculated with relative ease using MA. Hence, normalization of citation counts is feasible with MA. The citation analyses in MA and Scopus yield uniform results. The JNCS and the PR classes are similar in both databases, and, as a consequence, the evaluation of the researchers’ publication impact is congruent in MA and Scopus. Given the fast development in the last year, we postulate that MA has the potential to be used for full-fledged bibliometric analyses.

Hug, S.E., Ochsner, M. & Brändle, M.P. (2017) Citation analysis with microsoft academic. Scientometrics 111: 371.

Source: Citation analysis with microsoft academic

Completeness and overlap in open access systems: Search engines, aggregate institutional repositories and physics-related open sources

Authors: Tsay M-y, Wu T-l & Tseng L-l

Comment: This article compares several open access search engines (i.e., Google Scholar (GS), Microsoft Academic (MSA), OAIster, OpenDOAR, and Astrophysics Data System (ADS)) using publications of Nobel Laureates for Physics from 2001 to 2013. A short literature on comparing search engines is given. Both internal and external overlaps are studied. At the time of this work, GS had the highest coverage of this sample, but had a very high percentage of internal overlap (>92%). It actually covers all items in other sources, except for MSA. ADS and MSA both had coverage just below GS, with ADS having the lowest internal overlap of the three (just slightly higher than, which had 0 internal overlap).

Abstract: This study examines the completeness and overlap of coverage in physics of six open access scholarly communication systems, including two search engines (Google Scholar and Microsoft Academic), two aggregate institutional repositories (OAIster and OpenDOAR), and two physics-related open sources ( and Astrophysics Data System). The 2001–2013 Nobel Laureates in Physics served as the sample. Bibliographic records of their publications were retrieved and downloaded from each system, and a computer program was developed to perform the analytical tasks of sorting, comparison, elimination, aggregation and statistical calculations. Quantitative analyses and cross-referencing were performed to determine the completeness and overlap of the system coverage of the six open access systems. The results may enable scholars to select an appropriate open access system as an efficient scholarly communication channel, and academic institutions may build institutional repositories or independently create citation index systems in the future. Suggestions on indicators and tools for academic assessment are presented based on the comprehensiveness assessment of each system.

Tsay M-y, Wu T-l, Tseng L-l (2017) Completeness and overlap in open access systems: Search engines, aggregate institutional repositories and physics-related open sources. PLoS ONE 12(12): e0189751.

Source: Completeness and overlap in open access systems: Search engines, aggregate institutional repositories and physics-related open sources

Microsoft Academic is one year old: the Phoenix is ready to leave the nest

Authors: Harzing, AW. & Alakangas, S.

Comment: This is the third of a series of articles, by the first author, investigating the relative citation and publication coverage of Microsoft Academic (MA) within its first year of (re-)launch. Although the studies were of relatively small scale (citation record of 1 and 145 academics), they provided strong evidence for the advantages of MA over other databases. In particular, it possesses high coverage like Google Scholar and, at the same time, structured metadata like in Scopus and Web of Science. These, together with its fast growth, make MA an excellent alternative for bibliometrics and scientometrics studies.

Abstract: We investigate the coverage of Microsoft Academic (MA) just over a year after its re-launch. First, we provide a detailed comparison for the first author’s record across the four major data sources: Google Scholar (GS), MA, Scopus and Web of Science (WoS) and show that for the most important academic publications, journal articles and books, GS and MA display very similar publication and citation coverage, leaving both Scopus and WoS far behind, especially in terms of citation counts. A second, large scale, comparison for 145 academics across the five main disciplinary areas confirms that citation coverage for GS and MA is quite similar for four of the five disciplines. MA citation coverage in the Humanities is still substantially lower than GS coverage, reflecting MA’s lower coverage of non-journal publications. However, we shouldn’t forget that MA coverage for the Humanities still dwarfs coverage for this discipline in Scopus and WoS. It would be desirable for other researchers to verify our findings with different samples before drawing a definitive conclusion about MA coverage. However, based on our current findings we suggest that, only one year after its re-launch, MA is rapidly become the data source of choice; it appears to be combining the comprehensive coverage across disciplines, displayed by GS, with the more structured approach to data presentation, typical of Scopus and WoS. The Phoenix seems to be ready to leave the nest, all set to start its life into an adulthood of research evaluation.

Harzing, AW. & Alakangas, S. (2017) Microsoft Academic is one year old: the Phoenix is ready to leave the nest. Scientometrics 112: 1887.

Source: Microsoft Academic is one year old: the Phoenix is ready to leave the nest | Springer for Research & Development

Open Access: An Evaluation of its Impact, Obstacles, and Advancements

Author: Rachel A. Miles

Comments: A detailed article providing reviews on OA and Impact Metrics, and discussions on their misconceptions and misunderstandings. A review on OA mandates and policies is also provided. Other interesting discussions include those on Altmetrics, Eigenfactor, SNIP, JOI. An extensive list of potentially useful references are given.

Abstract: Access to research results is imperative in today’s robust digital age, yet access is often prevented by publisher paywalls. Open Access (OA) is the simple idea that all research should be free for all to access, use, and build upon. This paper will focus on three critical areas of the OA landscape: its impact on scholarship and the public, the obstacles to be overcome, and its advancements. The impact of OA actions and initiatives has been difficult to quantify, but the growing number of studies on OA have shown mostly overwhelmingly positive results. Cultural norms within academia, such as the reliance on the journal Impact Factor (IF) to assess the quality of individual research articles, have impeded the progress of OA. Conversely, federal mandates and institutional policies have supported the OA movement by requiring that scholarly publications be deposited into institutional or subject repositories immediately following publication. As information professionals, library and information science (LIS) professionals have a responsibility as practitioners, authors, and editors to support OA and encourage other academics to do the same.

Cite as: Miles, Rachel. (2016). Open Access: An Evaluation of its Impact, Obstacles, and Advancements. Bibliotekar, 58: (1-2).

Source: Open Access: An Evaulation of its Impact, Obstacles, and Advancements

A new methodology for comparing Google Scholar and Scopus

Authors: Henk F.Moed, Judit Bar-Ilan & Gali Halevi

Comments: This article is a small sample case study comparing meta data from Google Scholar and Scopus. Although the study only covers 36 articles in 12 journals (and the resulting ~7000 citations), it proposes some interesting methodologies. In particular, the methods for dealing with match-merging, citation duplicates and indexing speed may be of interest.

Abstract: A new methodology is proposed for comparing Google Scholar (GS) with other citation indexes. It focuses on the coverage and citation impact of sources, indexing speed, and data quality, including the effect of duplicate citation counts. The method compares GS with Elsevier’s Scopus, and is applied to a limited set of articles published in 12 journals from six subject fields, so that its findings cannot be generalized to all journals or fields. The study is exploratory, and hypothesis generating rather than hypothesis-testing. It confirms findings on source coverage and citation impact obtained in earlier studies. The ratio of GS over Scopus citation varies across subject fields between 1.0 and 4.0, while Open Access journals in the sample show higher ratios than their non-OA counterparts. The linear correlation between GS and Scopus citation counts at the article level is high: Pearson’s R is in the range of 0.8–0.9. A median Scopus indexing delay of two months compared to GS is largely though not exclusively due to missing cited references in articles in press in Scopus. The effect of double citation counts in GS due to multiple citations with identical or substantially similar meta-data occurs in less than 2% of cases. Pros and cons of article-based and what is termed as concept-based citation indexes are discussed.

Cite as: Moed HF, Bar-Ilan J & Halevi G (2016) A new methodology for comparing Google Scholar and Scopus. Journal of Informetrics 10(2): 533-551.

Source: A new methodology for comparing Google Scholar and Scopus

On impact factors and university rankings: from birth to boycott

Authors: Konstantinos I. Stergiou & Stephan Lessenich

Comments: This is a short article giving a quick literature review and a summary of criticisms on impact factors and university rankings.

Abstract: In this essay we explore parallels in the birth, evolution and final ‘banning’ of journal impact factors (IFs) and university rankings (URs). IFs and what has become popularized as global URs (GURs) were born in 1975 and 2003, respectively, and the obsession with both ‘tools’ has gone global. They have become important instruments for a diverse range of academic and higher education issues (IFs: e.g. for hiring and promoting faculty, giving and denying faculty tenure, distributing research funding, or administering institutional evaluations; URs: e.g. for reforming university/department curricula, faculty recruitment, promotion and wages, funding, student admissions and tuition fees). As a result, both IFs and GURs are being heavily advertised—IFs in publishers’ webpages and GURs in the media as soon as they are released. However, both IFs and GURs have been heavily criticized by the scientific community in recent years. As a result, IFs (which, while originally intended to evaluate journals, were later misapplied in the evaluation of scientific performance) were recently ‘banned’ by different academic stakeholders for use in ‘evaluations’ of individual scientists, individual articles, hiring/promotion and funding proposals. Similarly, URs and GURs have also led to many boycotts throughout the world, probably the most recent being the boycott of the German ‘Centrum fuer Hochschulentwicklung’ (CHE) rankings by German sociologists. Maybe (and hopefully), the recent banning of IFs and URs/GURs are the first steps in a process of academic self-reflection leading to the insight that higher education must urgently take control of its own metrics.

Cite As: Stergiou KI & Lessenich S (2014) On impact factors and university rankings: from birth to boycott. Ethics Sci Environ Polit 13:101-111.

Source: Inter Research » ESEP » v13 » n2 » p101-111

Scholarly publications beyond pay-walls: increased citation advantage for open publishing

Author: Susanne Mikki

Comments: This article suggests clear citation advantage for OA publishing, though the analysis was restricted to scholarly articles and ignored the legal status of full-texts.

Abstract: First, we aim to determine the total amount of scholarly articles freely available on the internet. Second, we aim to prove whether there exists a citation advantage for open publishing. The total scholarly publication output of Norway is indexed in Cristin, the Current Information System in Norway. Based on these data, we searched Google Scholar by either DOIs or titles and denoted a document as open available (OAv), when a link to a full-text was provided. We analysed the extracted data by publishing year, citations, availability and provider. Based on additional information indexed in Cristin, we furthermore analysed the data by year, institution, publisher and discipline. We find that the total share of freely available articles is 68%. Articles not available belong to prestigious publishers such as Elsevier, Springer, Routledge and Universitetsforlaget (the largest Norwegian academic publisher), which may be particularly essential for scholars worldwide. The largest provider, according to Google Scholar’s main link provision, is ResearchGate. In addition, institutional repositories play a major role in posting free article versions. Articles belonging to natural sciences and technology, and medicine and health were more likely to be open than articles belonging to the social sciences and humanities. Their respective OAv-shares are 72, 58 and 55%. We find a clear citation advantage for open publishing; on average, these documents received twice as many citations, indicating that open access is the future in publishing. This study is limited to scholarly articles only. Books and book chapters, which are usual publication formats for the humanities and social sciences, are excluded. Results do therefore not adequately reflect the situation for these disciplines. Furthermore, this study is limited to documents freely available on the internet, independent of the “legal” status of the posted full-text. With the data at hand, we were not able to distinguish between gold, green, hybrid, purely pay-walled and illicitly posted documents. Usually, articles indexed in Web of Science or SCOPUS are objects of investigation. However, these databases do not sufficiently cover the humanities and social sciences, and therefore cannot be representative of the total scholarly article output. This study captures the total article output of a country, independent on discipline and provides new insight into open publishing.

Mikki, S. (2017) Scholarly publications beyond pay-walls: increased citation advantage for open publishing. Scientometrics, 113(3): 1529-1538.

Source: Scholarly publications beyond pay-walls: increased citation advantage for open publishing | Springer for Research & Development

Imagining a Gold Open Access Future: Attitudes, Behaviors, and Funding Scenarios among Authors of Academic Scholarship

Authors: Carol Tenopir, Elizabeth D. Dalton, Lisa Christian, Misty K. Jones, Mark McCabe, MacKenzie Smith, Allison Fish

Comments: A survey on attitudes towards gold OA. The six main takeaways identified in the article were: 1. The prevailing attitude toward open access is often ambivalence. 2. Faculty are often conservative in their acceptance of OA. 3. Applied STEM fields are more accepting of OA. 4. Willingness to pay varies by source of funding. 5. Ambivalence provides a teachable moment for libraries. 6. Funding for APCs can be crucial for libraries.

Abstract: The viability of gold open access publishing models into the future will depend, in part, on the attitudes of authors toward open access (OA). In a survey of academics at four major research universities in North America, we examine academic authors’ opinions and behaviors toward gold OA. The study allows us to see what academics know and perceive about open access models, their current behavior in regard to publishing in OA, and possible future behavior. In particular, we gauge current attitudes to examine the perceived likelihood of various outcomes in an all-open access publishing scenario. We also survey how much authors at these types of universities would be willing to pay for article processing charges (APCs) from different sources. Although the loudest voices may often be heard, in reality there is a wide range of attitudes and behaviors toward publishing. Understanding the range of perceptions, opinions, and behaviors among academics toward gold OA is important for academic librarians who must examine how OA serves their research communities, to prepare for an OA future, and to understand how OA impacts the library’s role.

Tenopir et al. (2017) Imagining a Gold Open Access Future: Attitudes, Behaviors, and Funding Scenarios among Authors of Academic Scholarship. College & Research Libraries 78(6): 824-843.

Source: Imagining a Gold Open Access Future: Attitudes, Behaviors, and Funding Scenarios among Authors of Academic Scholarship | Tenopir | College & Research Libraries

Imagining the “open” university: Sharing scholarship to improve research and education

Author: Erin C. McKiernan

Comment: An interesting article in which describes the author’s idea of an open university and suggests practices/interventions for it.

Abstract: Open scholarship, such as the sharing of articles, code, data, and educational resources, has the potential to improve university research and education as well as increase the impact universities can have beyond their own walls. To support this perspective, I present evidence from case studies, published literature, and personal experiences as a practicing open scholar. I describe some of the challenges inherent to practicing open scholarship and some of the tensions created by incompatibilities between institutional policies and personal practice. To address this, I propose several concrete actions universities could take to support open scholarship and outline ways in which such initiatives could benefit the public as well as institutions. Importantly, I do not think most of these actions would require new funding but rather a redistribution of existing funds and a rewriting of internal policies to better align with university missions of knowledge dissemination and societal impact.

McKiernan, EC (2017) Imagining the “open” university: Sharing scholarship to improve research and education. PLoS Biol 15(10): e1002614.

Source: Imagining the “open” university: Sharing scholarship to improve research and education

Evidence of Open Access of scientific publications in Google Scholar: a large-scale analysis

Authors: Alberto Martín-Martín, Rodrigo Costas, Thed van Leeuwen, Emilio Delgado López-Cózar

Comment: This articles made use of Google Scholar (GS) to access links to available full texts of articles and reviews (limited to those with DOIs) in Web of Science (for 2009 and 2014). A python script was used to query GS (across a pool of IP addresses off-campus) for each DOI in the sample. Extracting data from GS took 3 months. Sources that provided full texts were then classified using DOAJ (publishers), OpenROAR, ROAR (repositories) and CrossRef (open license). This included manually checking about 1000 hosts as well. These were combined to determine OA status of individual DOI. Data was processed in R. The results were summarised and compared across disciplines and countries. Also, the numbers were similar to other recent large-scale studies on OA status that used similar data sets. This article also gave a good review on the literature of OA publication, licensing and copyright issues.

Abstract: This article uses Google Scholar (GS) as a source of data to analyse Open  Access (OA) levels across all countries and fields of research. All articles and reviews with a DOI and published in 2009 or 2014 and covered by the three main citation indexes in the Web of Science (2,269,022 documents) were selected for study. The links to freely available versions of these documents displayed in GS were collected. To differentiate between more reliable (sustainable and legal) forms of access and less reliable ones, the data extracted from GS was combined with information available in DOAJ, CrossRef, OpenDOAR, and ROAR. This allowed us to distinguish the percentage of documents in our sample that are made OA by the publisher (23.1%, including Gold, Hybrid, Delayed, and Bronze OA) from those available as Green OA (17.6%), and those available from other sources (40.6%, mainly due to ResearchGate). The data shows an overall free availability of 54.6%, with important differences at the country and subject category levels. The data extracted from GS yielded very similar results to those found by other studies that analysed similar samples of documents, but employed different methods to find evidence of OA, thus suggesting a relative consistency among methods.

Martín-Martín, A., Costas, R., van Leeuwen, T., & Delgado López-Cózar, E. (2018). Evidence of Open Access of scientific publications in Google Scholar: a large-scale analysis.

Source: Evidence of Open Access of scientific publications in Google Scholar: a large-scale analysis