University rankings: What do they really show? | SpringerLink

Author: Jill Johnes

Comment: This article highlights some issues with university rankings using correlation analysis and principal component analysis. Furthermore, it suggests an alternative weighting method that uses data envelopment analysis (DEA) that allows the weights to be optimised for individual institutions. DEA can further be used to derive performance groupings, rather than the usual point estimates. The ideas in this article are closely related to some of the discussions we  already had in the research team. Perhaps we may want to consider our data representation/aggregation along something like this, but with further specified min weight for each indicator?

Abstract: University rankings as developed by the media are used by many stakeholders in higher education: students looking for university places; academics looking for university jobs; university managers who need to maintain standing in the competitive arena of student recruitment; and governments who want to know that public funds spent on universities are delivering a world class higher education system. Media rankings deliberately draw attention to the performance of each university relative to all others, and as such they are undeniably simple to use and interpret. But one danger is that they are potentially open to manipulation and gaming because many of the measures underlying the rankings are under the control of the institutions themselves. This paper examines media rankings (constructed from an amalgamation of variables representing performance across numerous dimensions) to reveal the problems with using a composite index to reflect overall performance. It ends with a proposal for an alternative methodology which leads to groupings rather than point estimates.

Source: University rankings: What do they really show? | SpringerLink

Executive power and scaled-up gender subtexts in Australian entrepreneurial universities: Gender and Education: Vol 27, No 3

Authors: Jill Blackmore and  Naarah Sawers, 2015

Comment: This 3 year ARC funded study researched higher echelons of management in three self-identified “entrpreneurial’ Australian universities. They identified gender “subtexts” in the restructuring of universities to meet global markets rather than a national focus, with emphases on science and technology in research, management and obsession with rankings. Despite better gender equality overall, more women are in casual positions with difficulties reaching tenured and leadership positions.

Deputy Vice Chancellor and Pro Vice Chancellor positions have proliferated in response to the global, corporatised university landscape [Scott, G., S. Bell, H. Coates, and L. Grebennikov. 2010. “Australian Higher Education Leaders in Times of Change: The Role of Pro Vice Chancellor and Deputy Vice Chancellor.” Journal of Higher Education Policy and Management 32 (4): 401–418]. Senior leadership is the sphere where academic and management identities are negotiated and values around the role of the university are decided. This paper examines the changing and gendered nature of the senior leadership setting and its implications for diversity in and of university leadership. The analysis draws from a three-year empirical study funded by the Australian Research Council on leadership in Australian universities. It focuses on executive leaders in three universities – one which is research-intensive, the second, in a regional site, and the third, university of technology. The article argues that the university landscape and its management systems are being restructured in gendered ways. It utilises the notion of organisational gender subtexts to make explicit how gender works through structural and cultural reform.


Source: Executive power and scaled-up gender subtexts in Australian entrepreneurial universities: Gender and Education: Vol 27, No 3

Empirical analysis and classification of database errors in Scopus and Web of Science – ScienceDirect

Authors: Franceschini F, Maisano D & Mastrogiacomo L

Comment: This is an article studying various errors in Scopus and Web of Science (WoS) databases. These include citation indexing errors, missing links, missing DOIs, incorrect author names, etc. Manual check was done on a sample of errors. After classification of errors, it found that the distributions of errors were very different between Scopus and WoS.

Abstract: In the last decade, a growing number of studies focused on the qualitative/quantitative analysis of bibliometric-database errors. Most of these studies relied on the identification and (manual) examination of relatively limited samples of errors.

Using an automated procedure, we collected a large corpus of more than 10,000 errors in the two multidisciplinary databases Scopus and Web of Science (WoS), mainly including articles in the Engineering-Manufacturing field. Based on the manual examination of a portion (of about 10%) of these errors, this paper provides a preliminary analysis and classification, identifying similarities and differences between Scopus and WoS.

The analysis reveals interesting results, such as: (i) although Scopus seems more accurate than WoS, it tends to forget to index more papers, causing the loss of the relevant citations given/obtained, (ii) both databases have relatively serious problems in managing the so-called Online-First articles, and (iii) lack of correlation between databases, regarding the distribution of the errors in several error categories.

The description is supported by practical examples concerning a variety of errors in the Scopus and WoS databases.

Source: Empirical analysis and classification of database errors in Scopus and Web of Science – ScienceDirect

Gender equality in academia: a critical reflection: Journal of Higher Education Policy and Management: Vol 37, No 3

AUTHORS: Hilary P.M. Winchester & Lynette Browning, Central Queensland University

The article reviews growth in women academics in Australian universities from the 1980s to 2014 when overall percentage of women reached 44%. However, the share of women in senior positions is still lower (31%) than men’s.  The authors identify an under representation of women in research and research funding. The article includes and discusses milestones of legislation, policy, research and strategic programs,  strategies and frameworks that have worked.

Gender equality in academia has been monitored in Australia for the past three decades so it is timely to reflect on what progress has been made, what works, and what challenges remain. When data were first published on the gender composition of staff in Australian universities in the mid-1980s women comprised 20 per cent of academic staff and held 6 per cent of senior positions. Since the early 1990s many Australian universities have had policies in place to remove sex discrimination and initiatives to increase women’s representation in academia. Two decades on, women comprise 44 per cent of academic staff and hold 31 per cent of senior positions. How did this happen? What worked? Are there still challenges to be addressed? This paper provides a critical reflection on what has worked, the practical impacts on gender in academia in Australia and what challenges remain for the future.

Source: Gender equality in academia: a critical reflection: Journal of Higher Education Policy and Management: Vol 37, No 3

Does staff diversity imply openness to diversity? | International Journal of Educational Management | Vol 27, No 6

AUTHOR: Jakob Lauring, Jan Selmer; Aarhus University Denmark

COMMENT: This is a really interesting exploration and correlation of different dimensions of  staff diversity and openness to diversity in 3 large Danish universities. The authors differentiate between surface level, demographic diversity (age, gender) and deep-level, international diversity (cultural and linguistic) and found these had different associations with attitudes and openness to value, visible, informational and linguistic diversity in the workplace. Demographic diversity had a negative association whereas international diversity was more positive. Almost 3/4 of the respondents were male.


Post‐secondary educational organizations are currently some of the most diverse settings to be found. However, few educational studies have dealt with staff diversity and hardly any has looked outside the USA. The purpose of this paper is to present a study of members of international university departments in Denmark. The authors set out to investigate the relationship between different types of staff diversity and openness to diversity in terms of linguistic, visible, value, and informational heterogeneity.
This study uses responses from 489 staff members from diverse university departments to a self‐report electronic survey.
It was found that diversity‐related internationalization (cultural and linguistic) was generally positively related to favorable diversity attitudes. Inherent demographic diversity (age and gender), on the other hand, was unrelated or negatively associated with positive diversity attitudes.

Source: Does staff diversity imply openness to diversity? | International Journal of Educational Management | Vol 27, No 6 2013

Underrepresented & Underpaid Diversity & Equity Among Canada’s Post-Secondary Education Teachers

Canadian Association of University Teachers
Association canadienne des professeures et professeurs d’université

This reports on slow progress in equity and diversity in Canadian universities, particularly in the employment of “racialised minorities” and Aboriginal staff members, and wage differences. This is occurring despite equity and diversity policies in place. The report also notes limited availability of data and uses mostly Statistics Canada 2016 census data. Will be useful for Australian comparison.


While post-secondary institutions are publicly committed to equity and diversity, progress has been slow in achieving employment and wage equity for academic staff.
This report provides a snapshot of academic staff representation and income in Canada’s universities and colleges in 2016, noting in particular changes in the university sector over the last decade.
While available data is limited, it does reveal that the academic workforce is not as diverse as either the student body or the labour force. Evidence also shows significant wage gaps: between men and women; and between white, Aboriginal and racialized academic staff.

Click to access caut_equity_report_2018-04final.pdf

Elsevier journals — some facts

Author: Timothy Gower
Blogpost April 24, 2014

Comment: This long blog post discusses the author’s attempts, successful in many cases, to obtain the costs of Elsevier journal subscriptions at the UK Russell Group of universities. It includes some amusing detailed correspondence with JISC and the universities. Also  related discussion around APCs and their impact on subscription costs, Elsevier costs in some US universities, Brazil. Also in the post and related comments are some useful data sources and related analysis.

Introduction: A little over two years ago, the Cost of Knowledge boycott of Elsevier journals began. Initially, it seemed to be highly successful, with the number of signatories rapidly reaching 10,000 and including some very high-profile researchers, and Elsevier making a number of concessions, such as dropping support for the Research Works Act and making papers over four years old from several mathematics journals freely available online. It has also contributed to an increased awareness of the issues related to high journal prices and the locking up of articles behind paywalls….

I  have come to the conclusion that if it is not possible to bring about a rapid change to the current system, then the next best thing to do, which has the advantage of being a lot easier, is to obtain as much information as possible about it. Part of the problem with trying to explain what is wrong with the system is that there are many highly relevant factual questions to which we do not yet have reliable answers.

Elsevier journals — some facts

Open Educational Resources and Rhetorical Paradox in the Neoliberal Univers(ity) | Journal of Critical Library and Information Studies


A critique of the aims/achievements of OER in the context of openness and inclusion, equity debates. The author argues that OER is still part of and replicates neo-liberal educations systems and is not yet disruptive. This may be possible if OER can be developed at local levels involving students and educators and those who do not currently have an opportunity to produce knowledge for educational purposes.

Author: Nora Almeida,
New York City College of Technology, CUNY

Keywords: Open Educational Resources, Social Justice, Neoliberalism, Pedagogy, Information Access, Digital Education


As a phenomenon and a quandary, openness has provoked conversations about inequities within higher education systems, particularly in regards to information access, social inclusion, and pedagogical practice. But whether or not open education can address these inequities, and to what effect, depends on what we mean by “open” and specifically, whether openness reflexively acknowledges the fraught political, economic, and ethical dimensions of higher education and of knowledge production processes. This essay explores the ideological and rhetorical underpinnings of the open educational resource (OER) movement in the context of the neoliberal university. This essay also addresses the conflation of value and values in higher education – particularly how OER production processes and scholarship labor are valued. Lastly, this essay explores whether OER initiatives provide an opportunity to reimagine pedagogical practices, to reconsider authority paradigms, and potentially, to dismantle and redress exclusionary educational practices in and outside of the classroom. Through a critique of neoliberalism as critically limiting, an exploration of autonomy, and a refutation of the precept that OER can magically solve social inequalities in higher education, the author ultimately advocates for a reconsideration of OER in context and argues that educators should prioritize conversations about what openness means within their local educational communities.

Author Biography

Nora Almeida, New York City College of Technology, CUNY

Source: Open Educational Resources and Rhetorical Paradox in the Neoliberal Univers(ity) | Journal of Critical Library and Information Studies

A new methodology for comparing Google Scholar and Scopus

Authors: Henk F.Moed, Judit Bar-Ilan & Gali Halevi

Comments: This article is a small sample case study comparing meta data from Google Scholar and Scopus. Although the study only covers 36 articles in 12 journals (and the resulting ~7000 citations), it proposes some interesting methodologies. In particular, the methods for dealing with match-merging, citation duplicates and indexing speed may be of interest.

Abstract: A new methodology is proposed for comparing Google Scholar (GS) with other citation indexes. It focuses on the coverage and citation impact of sources, indexing speed, and data quality, including the effect of duplicate citation counts. The method compares GS with Elsevier’s Scopus, and is applied to a limited set of articles published in 12 journals from six subject fields, so that its findings cannot be generalized to all journals or fields. The study is exploratory, and hypothesis generating rather than hypothesis-testing. It confirms findings on source coverage and citation impact obtained in earlier studies. The ratio of GS over Scopus citation varies across subject fields between 1.0 and 4.0, while Open Access journals in the sample show higher ratios than their non-OA counterparts. The linear correlation between GS and Scopus citation counts at the article level is high: Pearson’s R is in the range of 0.8–0.9. A median Scopus indexing delay of two months compared to GS is largely though not exclusively due to missing cited references in articles in press in Scopus. The effect of double citation counts in GS due to multiple citations with identical or substantially similar meta-data occurs in less than 2% of cases. Pros and cons of article-based and what is termed as concept-based citation indexes are discussed.

Cite as: Moed HF, Bar-Ilan J & Halevi G (2016) A new methodology for comparing Google Scholar and Scopus. Journal of Informetrics 10(2): 533-551.

Source: A new methodology for comparing Google Scholar and Scopus

On impact factors and university rankings: from birth to boycott

Authors: Konstantinos I. Stergiou & Stephan Lessenich

Comments: This is a short article giving a quick literature review and a summary of criticisms on impact factors and university rankings.

Abstract: In this essay we explore parallels in the birth, evolution and final ‘banning’ of journal impact factors (IFs) and university rankings (URs). IFs and what has become popularized as global URs (GURs) were born in 1975 and 2003, respectively, and the obsession with both ‘tools’ has gone global. They have become important instruments for a diverse range of academic and higher education issues (IFs: e.g. for hiring and promoting faculty, giving and denying faculty tenure, distributing research funding, or administering institutional evaluations; URs: e.g. for reforming university/department curricula, faculty recruitment, promotion and wages, funding, student admissions and tuition fees). As a result, both IFs and GURs are being heavily advertised—IFs in publishers’ webpages and GURs in the media as soon as they are released. However, both IFs and GURs have been heavily criticized by the scientific community in recent years. As a result, IFs (which, while originally intended to evaluate journals, were later misapplied in the evaluation of scientific performance) were recently ‘banned’ by different academic stakeholders for use in ‘evaluations’ of individual scientists, individual articles, hiring/promotion and funding proposals. Similarly, URs and GURs have also led to many boycotts throughout the world, probably the most recent being the boycott of the German ‘Centrum fuer Hochschulentwicklung’ (CHE) rankings by German sociologists. Maybe (and hopefully), the recent banning of IFs and URs/GURs are the first steps in a process of academic self-reflection leading to the insight that higher education must urgently take control of its own metrics.

Cite As: Stergiou KI & Lessenich S (2014) On impact factors and university rankings: from birth to boycott. Ethics Sci Environ Polit 13:101-111.

Source: Inter Research » ESEP » v13 » n2 » p101-111