Category: Assessment

Empirical analysis and classification of database errors in Scopus and Web of Science – ScienceDirect

Authors: Franceschini F, Maisano D & Mastrogiacomo L

Comment: This is an article studying various errors in Scopus and Web of Science (WoS) databases. These include citation indexing errors, missing links, missing DOIs, incorrect author names, etc. Manual check was done on a sample of errors. After classification of errors, it found that the distributions of errors were very different between Scopus and WoS.

Abstract: In the last decade, a growing number of studies focused on the qualitative/quantitative analysis of bibliometric-database errors. Most of these studies relied on the identification and (manual) examination of relatively limited samples of errors.

Using an automated procedure, we collected a large corpus of more than 10,000 errors in the two multidisciplinary databases Scopus and Web of Science (WoS), mainly including articles in the Engineering-Manufacturing field. Based on the manual examination of a portion (of about 10%) of these errors, this paper provides a preliminary analysis and classification, identifying similarities and differences between Scopus and WoS.

The analysis reveals interesting results, such as: (i) although Scopus seems more accurate than WoS, it tends to forget to index more papers, causing the loss of the relevant citations given/obtained, (ii) both databases have relatively serious problems in managing the so-called Online-First articles, and (iii) lack of correlation between databases, regarding the distribution of the errors in several error categories.

The description is supported by practical examples concerning a variety of errors in the Scopus and WoS databases.

Source: Empirical analysis and classification of database errors in Scopus and Web of Science – ScienceDirect

Dimensions: A competitor to Scopus and the Web of Science? – ScienceDirect

Author: Mike Thelwall

Comment: This articles compares samples of data from Scopus with Dimensions. It found that the citation counts in Dimensions are in line with those obtained in Scopus. The slightly lower numbers in citations give indication that the coverage of Dimensions may not be much greater than that of Scopus. Although, this is not explicitly checked.

Abstract: Dimensions is a partly free scholarly database launched by Digital Science in January 2018. Dimensions includes journal articles and citation counts, making it a potential new source of impact data. This article explores the value of Dimensions from an impact assessment perspective with an examination of Food Science research 2008–2018 and a random sample of 10,000 Scopus articles from 2012. The results include high correlations between citation counts from Scopus and Dimensions (0.96 by narrow field in 2012) as well as similar average counts. Almost all Scopus articles with DOIs were found in Dimensions (97% in 2012). Thus, the scholarly database component of Dimensions seems to be a plausible alternative to Scopus and the Web of Science for general citation analyses and for citation data in support of some types of research evaluations.

Source: Dimensions: A competitor to Scopus and the Web of Science? – ScienceDirect

The prevalence of green and grey open access: Where do physical science researchers archive their publications? | SpringerLink

Authors: Li Zhang & Erin Watson

Comment: This paper focuses on comparing green and grey (archiving in academic social media or personal/departmental website) OA, for CIHR funded research. Data is extract from WoS and Google Scholar used to determine green and grey OA. The prevalence of grey OA is highlighted, and the low up-take of green OA is shown as not attributed to publisher policies, as most do allow green. The takeaway is suggestion to rethink about ways to archive OA, given the high costs of running an institutional repository.

Abstract: The Canadian Institute of Health Research (CIHR) implemented an open access policy for its grant recipients in 2008. We used bibliographic data from the Web of Science to find out how CIHR-funded researchers in the physical sciences self-archived their publications. We also examined the self-archiving policies of the journals in which the researchers published, and compared the citation rates of two different self-archiving approaches: the green open access route (deposit in an institutional or subject repository) and the grey open access route (deposit in an academic social network or personal/departmental website). Only 14% of the articles were openly accessible through the green open access route, while 37% could be accessed through the grey open access route. We cannot ascribe the low uptake of green open access to publishers’ self-archiving policies, as almost all journals allowed self-archiving through the green open access route. Authors deposited 31% of their publications in ResearchGate, the most popular self-archiving option in our study, while they deposited only 2.1% of their publications in institutional repositories, the least popular option. The citation rates of the various self-archiving approaches did not differ significantly. Our results suggest that it may be time to rethink how to achieve open access.

Source: The prevalence of green and grey open access: Where do physical science researchers archive their publications? | SpringerLink

Article processing charge (APC) for publishing open access articles: the Brazilian scenario | SpringerLink

Authors: Cleusa Pavan & Marcia C. Barbosa

Comment: This article provides a detailed literature on OA publication in Brazil and tracks the progress over time. This is splits among journals with different APC policies. Again, the caveat may be the sole use of Web of Science data. However, it is still interesting to note that, although the number of OA publications has increased (in absolute terms, rather than percentages), the output process/vehicle has also become endogenic locally. This is possibly driven by lack of funds. The authors suggest policies for funding APC are required to increase international publications.

Abstract: The article processing charge (APC) provides economic sustainability for scientific journals that publish in open access (OA). In this work, documents published in OA between 2012 and 2016 by authors with Brazilian affiliation are identified, the profile of these publications is analyzed and the cost of APC is estimated. In order to do so, data from 930 journals and 63,847 documents were collected from the Web of Science Core Collection. It was found that 59% of these documents were published in journals that charge APC. The total expenditures for the 5-year period were estimated at approximately USD 36 million, the weighted average cost per document at USD 957.75 and the average cost per journal at USD 1492.27. The profile of these publications shows that journals indexed by SciELO represent 67% of the 63,847 documents. The use of mega-journals increased over the period, which implies an increase in expenditure in publications, since the average APC per journal was USD 2059.77. It was observed that the OA Brazilian scientific production is characterized by an endogenic profile and has a preference for the Gold road with APC. These results suggest that policies for funding charges are required to stimulate a more international attitude.

Source: Article processing charge (APC) for publishing open access articles: the Brazilian scenario | SpringerLink

The History, Deployment, and Future of Institutional Repositories in Public Universities in South Africa – ScienceDirect

Author: Siviwe Bangani

Comment:
Another interesting paper about IRs in South Africa (SA). Web data was collected, together with interviews been conducted. A detailed history of IRs in SA is given. While many of the South African universities have signed various international declarations and initiative on OA, they often don’t have an institutional policy on OA. Various factors (obstacles and enablers) are listed. Amount of funding is relatively low compared to other countries. Varying IR sizes, types of objects in IRs, multiple language support and issues, and suggestions for development are presented and discussed.

Abstract:
This paper investigates the history, deployment, and content of institutional repositories (IRs) in public universities in South Africa. Some of the local, national and international drivers and enablers that ensure the establishment and survival of the institutional repositories are identified. Lastly, an attempt is made to determine the future of the IRs. Findings include that South African universities were among the first universities in the world to host IRs with the first IR established in 2000. The most prevalent and dominant content in South African public university collections are electronic theses and dissertations (ETDs). There are signs that this is changing as more libraries cover research outputs emanating from the universities. African languages are sparsely represented in IRs in South Africa. The majority of universities in the country signed the Berlin Declaration on Open Access to Knowledge in the Sciences and Humanities, and the Budapest Open Access Initiative. Many of them do not have their own open access policy. The driving factors include the decline in government subsidy, increase in journal subscriptions, depreciation of the South African currency, and addition of the Value Added Tax (VAT) of 14% on electronic resources by the South Africa taxman while the enabling factors include the international open access mandates, the Carnegie Foundation grants, and the National Research Foundation’s statement on open access.

Bangani S (2018) The History, Deployment, and Future of Institutional Repositories in Public Universities in South Africa. The Journal of Academic Librarianship 44(1): 39-51.

Source: The History, Deployment, and Future of Institutional Repositories in Public Universities in South Africa – ScienceDirect

Institutional Repositories in Chinese Open Access Development: Status, Progress, and Challenges – ScienceDirect

Authors: Jing Zhong & Shuyong Jiang

Comment: An interesting paper interrogating institutional repositories (IR) in China. These IRs were accessed via ROAR, OpenDOAR, SouOA and CHAIR, though many URL links were broken. The article highlighted the slow development of OA repositories in China and attributed this to the lack of policy and support at all levels. At the end of the article, it mentioned that the Chinese Academy of Sciences and the National Natural Science Foundation of China, in May 2014, released an Open Access policy statement requiring that its funded research papers be made open access in IRs within 12 months after their publication. It would be interesting to follow-up on whether this had made any significant impact.

Abstract:
Open Access (OA) movement in China is developing with its own track and speed. Compared to its western counterparts, it moves slowly. However, it keeps growing. More significantly, it provides open and free resources not only to Chinese scholars, but also to those of China studies around the world. The premise is whether we can find them in an easy and effective fashion. This paper will describe the status of the OA movement in China with a focus on institutional repositories (IR) in Chinese universities and research institutes. We will explore different IR service modules and discuss their coverage, strengths, limitation, and most importantly implications to the East Asian Collection in the US.

Zhong J & Jiang S (2016) Institutional Repositories in Chinese Open Access Development: Status, Progress, and Challenges. The Journal of Academic Librarianship 42(6): 739-744.

Source: Institutional Repositories in Chinese Open Access Development: Status, Progress, and Challenges – ScienceDirect

Virtual Issue: Measuring the Impact of Arts and Humanities Research in Europe | Research Evaluation | Oxford Academic

Authors: Authors of the articles include Clare Donovan, Sverker Sörlin, Ellen Hazelkorn, Andrew Gibson and others.

Summary: A virtual issue of Research Evaluation that covers the issue of impact from Art and Humanities research. In particular a review of the literature from Reale et al is likely useful for providing an overview of work that has been done.

Abstract: In recent years, the concept of ‘impact’, or the wider value of research for society, has climbed to the top of national and EU science policy agendas. Yet the way impact has been imagined often relates to concepts of value, and traditional measurements of impact, which emphasise cost-benefit analyses and the economic value of research, which are poorly suited to arts and humanities research (AHR). This virtual issue of Research Evaluation reflects distinct EU and national debates and characteristics important for exploring the societal value of AHR. These examples are useful for understanding AHR, and also the societal value of research in general.

Source: Virtual Issue: Measuring the Impact of Arts and Humanities Research in Europe | Research Evaluation | Oxford Academic

How significant are the public dimensions of faculty work in review, promotion, and tenure documents?

Authors:  Juan Pablo Alperin, Gustavo E. Fischman, Erin C. McKiernan, Carol Muñoz Nieves, Meredith T. Niles, Lesley Schimanski

Summary: The authors examine tenure and promotions requirements documents from 129 North American universities looking for evidence of interest in engagement, open access and other community/public focussed activities. Relatively little is found and the the word ‘community’ is often co-located with words dealing with professional and academic service suggesting an inward looking focus. Also interesting from its perspective on textual analysis as a tool for analysis.

Abstract: Much of the work of universities, even private institutions, has significant public dimensions. Faculty work in particular is often funded by public funds, is aimed at serving the public good, and is subject to public evaluation. To understand how the public dimensions of faculty work are valued, we analyzed review, tenure and promotion documents from a representative sample of 129 Canadian and American universities. We found terms and concepts related to public and community are mentioned in a large portion of documents, but mostly in ways that relate to service—an undervalued aspect of academic careers. Moreover, we find significant mentions of traditional research outputs and citation-based metrics. Such outputs and metrics reward faculty work targeted to academics, and mostly disregard the public dimensions. We conclude that institutions that want to live up to their public mission need to work towards systemic change in how faculty work is assessed and incentivized.

Alperin, J.P., Muñoz Nieves, C., Schimanski, L., Fischman, G.E., Niles, M.T. & McKiernan, E.C. (2018). How significant are the public dimensions of faculty work in review, promotion, and tenure documents? Humanities Commons [preprint]. https://doi.org/10.17613/M6W950N35

Source: How significant are the public dimensions of faculty work in review, promotion, and tenure documents? | hc:21015 | Humanities CORE

Elsevier journals — some facts

Leave a comment

Author: Timothy Gower
Blogpost April 24, 2014

Comment: This long blog post discusses the author’s attempts, successful in many cases, to obtain the costs of Elsevier journal subscriptions at the UK Russell Group of universities. It includes some amusing detailed correspondence with JISC and the universities. Also  related discussion around APCs and their impact on subscription costs, Elsevier costs in some US universities, Brazil. Also in the post and related comments are some useful data sources and related analysis.

Introduction: A little over two years ago, the Cost of Knowledge boycott of Elsevier journals began. Initially, it seemed to be highly successful, with the number of signatories rapidly reaching 10,000 and including some very high-profile researchers, and Elsevier making a number of concessions, such as dropping support for the Research Works Act and making papers over four years old from several mathematics journals freely available online. It has also contributed to an increased awareness of the issues related to high journal prices and the locking up of articles behind paywalls….

I  have come to the conclusion that if it is not possible to bring about a rapid change to the current system, then the next best thing to do, which has the advantage of being a lot easier, is to obtain as much information as possible about it. Part of the problem with trying to explain what is wrong with the system is that there are many highly relevant factual questions to which we do not yet have reliable answers.

You can leave a response, or trackback from your own site.

Elsevier journals — some facts

Evaluation of Openness in the Activities of Research Organisations and Research Funding Organisations in 2016

Leave a comment
Author: finland Ministry of Education and Culture, Open Science and Research Initiative

Notes: Interesting scoring of Finnish research organisations regarding progress towards openness using data retrieved from public websites. A follow up survey gave organisations the opportunity to correct and supplement data. Also compared with other European research organisations. Because of the rapidly changing landscape the assessment was not repeated in 2017.

Abstract: This evaluation of the openness of Finnish research performing and funding organisations was completed as part of the Open Science and Research Initiative (ATT) by the Ministry of Education and Culture. The target of this evaluation is to assess the openness of operational cultures in research organisations and research funding organisations. The key objectives, against which the assessments are made, are defined in the Open Science and Research Roadmap. More information about the evaluation can be found at openscience.fi/openculture

http://www.doria.fi/handle/10024/127273