Taming the wild West: a systematic review and evaluation of bibliometric reviews of educational research

Abstract

Bibliometric reviews have proliferated across academic disciplines in recent years. This unprecedented publication trajectory has prompted concerns about quality and the contributions of these reviews to knowledge. This systematic review used descriptive statistics and content analysis to review 1,873 bibliometric reviews of educational research published in Scopus-indexed journals through 2024. The authors analyzed the research landscape, topical trends, and methodological quality of these education reviews. Remarkably, 87% percent were published between 2020 and 2024, and 97% since 2015. A majority were published in Scopus Q1 and Q2 journals, including 49 in top 1% journals by field. Also surprising, 69% of the reviews were authored by scholars in emerging nations. This contrasts with original research publications, which have traditionally been dominated by authors from economically developed societies. The most frequently studied topics concerned higher education, research performance, and technology-enabled learning. Further analysis found that numerous occurrences of authors conducting reviews on the same topic, using similar methods, over the same time span, and without cross-referencing. A methodological quality rubric was applied to a stratified random sample of 60 bibliometric reviews published in high-, mid-, and low-percentile Scopus journals. Although these analyses identified some exemplary reviews, the broader results indicated that many bibliometric reviews failed to realize their potential due to persistent methodological weaknesses and a lack of conceptual depth. These deficits threaten the quality and contribution of bibliometric reviews in education and other disciplines. Specific recommendations are made to improve the quality and contributions of future bibliometric reviews.