Avalilação PREreview de Rewriting the Politics of Publication: Tracking 25 Years of Debate in Academic Reform with Bibliometric Analysis

Review of “Rewriting the Politics of Publication: Tracking 25 Years of Debate in Academic Reform with Bibliometric Analysis”

Discussion Participants (in random order): This review was the result of two live review sessions and a period of asynchronous review that included additional discussion participants Ava Chan, Mercury Shitindo, and Naaman Arodi.

Summary

This preprint investigates how scholarly discourse on academic publishing reform has evolved between 2000 and 2025, focusing on three central debates: editorial gatekeeping and bias, prestige-driven research metrics, and barriers/equity in access. Using bibliometric analysis of Scopus-indexed literature, supplemented by thematic mapping and time-slicing around milestones such as DORA and Sci-Hub, the study provides a longitudinal overview of how debates have shifted over the last 25 years.

Strengths

The preprint demonstrates several notable strengths that contribute to its value as a scholarly work. First, the research question is clearly articulated and thoughtfully framed around central debates of academic publishing reform, namely editorial gatekeeping, prestige-driven metrics, and barriers to equity in access. This clarity provides a strong foundation for the study and brings coherence to the analysis.

Second, the data analysis is substantial and systematically organized, with temporal slicing that allows for meaningful comparisons across key milestones such as DORA and Sci-Hub. This design choice enhances the paper’s ability to capture both long-term trends and more recent developments in the field. Methodologically, the use of bibliometric tools such as the Bibliometrix package in R, multiple correspondence analysis, and thematic mapping demonstrates technical sophistication. The authors also pay careful attention to data management.

Third, beyond technical execution, the paper engages directly with globally relevant debates about equity, access, and scholarly integrity, ensuring its relevance across disciplines and geographies.

Finally, the writing is clear and well-structured.

Major Issues

Despite its strengths, the manuscript also presents several important limitations that warrant careful attention.

At 57 pages, the paper is longer than most published journal articles. Even without that constraint, we found that the length contributed to a diffuse narrative. As reviewers, we considered two possible approaches to addressing this issue.

• The first approach would be to divide the paper into multiple articles, each focusing on editorial gatekeeping, prestige metrics, or equity and access. This approach would have the advantage of allowing each paper to be addressed to a specific audience.

• A second option would be to focus the paper on the narrative reflected in the thematic maps, especially their motor quadrants. This is where the narrative gains traction. With this approach, you would keep only the thematic maps and the thematic evolution diagrams in the main text of the article. Use the time-sliced maps where applicable, and consider combining the two evolution diagrams into one. You would then move the rest of the figures, many of the tables, and their analysis to supplemental materials. Although that may seem drastic, shortening the paper like that would greatly increase the number of journals that would consider the paper for publication.

We recommend adding a designated Limitations section in which some of the following concerns can be addressed.

A central limitation for us was the exclusive reliance on English-language literature within Scopus, which risks reproducing the very Global North and English-language bias that the study seeks to critique. We recognize that a comparative and multilingual analysis would have been a considerably more complex study. However, the absence of regional databases such as SciELO or AJOL, as well as open alternatives like OpenAlex, narrows the inclusivity of the analysis and should be more explicitly acknowledged and justified. Although Scopus indeed offers comprehensive metadata, it should be acknowledged that similar fields (e.g., citation details, bibliographic data, affiliations, and in some cases funding information) are also available in Web of Science (WoS). The claim that WoS metadata is typically restricted to authors, titles, sources, and abstracts does not reflect the full capabilities of the database. There are also other bibliometric data sources available, such as OpenAlex, an open, large-scale, and increasingly utilized alternative. Rather than framing the rationale solely as a comparison between Scopus and WoS, the authors may wish to articulate why Scopus was chosen over other possible sources and, if possible, describe how they would expect the results to be different if other databases had been included. Doing so would demonstrate a broader awareness of the expanding landscape of bibliographic databases and provide a more comprehensive justification for the methodological choice.

In addition, the treatment of historical milestones seems incomplete. While DORA and Sci-Hub are foregrounded, other equally significant developments, including the Leiden Manifesto, CoARA, and most notably the COVID-19 pandemic, are insufficiently integrated. This omission weakens the historical contextualization, particularly given the pandemic’s profound impact on scholarly publishing, the types of research that could be conducted, preprint adoption, and debates around access. As COVID-19 was one of the keywords that was excluded or combined, this should be referenced and explained in the main text.

Relatedly, the description of data processing should be more transparent. The processes of keyword cleaning, deduplication, and classification are not fully documented, leaving questions about justification for keyword removal and the handling of documents that span multiple thematic categories. Greater transparency, through the release of keyword dictionaries, detailed classification rules, and code, would significantly enhance reproducibility and confidence in the findings.

Concerns also arise in the interpretation of the multiple components analysis (MCA). In some instances, the first two dimensions explain less than 11% of the variance, raising doubts about the robustness of the clusters presented. Without additional evidence, such as reporting cumulative variance across more dimensions or validating clusters through alternative methods, the interpretive strength of these analyses remains limited. In the thematic maps, it would be helpful to look at the clusters in the emerging / declining quadrants at a more granular level and to clearly identify each one as either emerging or declining. As a reader, not knowing the direction of change distracts from the clarity of the narrative.

Finally, the paper is largely descriptive, while its title, “Rewriting the Politics” can be read as promising a more robust critique. A more precise framing such as “Tracing 25 Years of Debate” or “Mapping the Politics of Publication” may better capture the scope of the work. To the extent practical, engaging with theory in the Discussion section would add depth to the interpretation of your findings on the equity debates. Some relevant schools of thought to consider would be feminist theory, Bourdieu’s field theory, and decolonial and Global South epistemologies.

Minor Issues

In addition to these major concerns, there are some minor issues that, if addressed, would improve the readability, accessibility, and overall coherence of the manuscript.

With respect to tables and figures, our main recommendation is that as many as possible be moved to supplemental materials, especially if you are seeking publication at a journal that prints hard copies. Colour printing is especially costly, so it would be helpful to adapt any figures that will remain in the main text for high-resolution black-and-white reproduction. Providing interactive versions of figures online would be another option to consider. The current captions have the advantage of being consistent across each debate. However, you may want to revise captions to help each figure stand on its own to the extent practical, including by spelling out abbreviations. An option to make the figures easier to interpret, especially in black and white, would be to number the identified clusters.

Miscellaneous Remarks

There are also some broader considerations that would enhance the overall impact and relevance of the manuscript.

• The discussion would be more compelling if bibliometric patterns were more explicitly linked to policy outcomes, for instance, how trends in editorial control or research metrics have influenced funding distribution, institutional evaluation systems, or open access mandates.

• The Conclusion could be strengthened by situating the findings more clearly within current policy contexts, which have changed significantly even since this research was conducted. Acknowledging recent U.S. policy shifts that restrict equity-driven research, such as the 2025 NSF directives, would underline the continued political stakes of debates in academic publishing and make the study resonate more strongly with present-day developments.

Context and Contribution

The preprint is a valuable and timely contribution to scholarship on publishing reform, offering a long-term bibliometric mapping of critical debates. It is especially timely in that 2025 is likely to represent another major shift. The study’s novelty lies in synthesizing 25 years of discourse and situating them against reform milestones. We believe that this paper contains the makings of at least one paper that we would recommend for publication. As a newly formed PREreview Club, we found it especially motivating to find in the paper’s conclusions “the persistence of identity-based and structural bias necessitates a reformation from ‘Gatekeeper’ to ‘Mentor’” (p. 48) and “all scholars were once learners who benefited from the patience and guidance of their predecessors; extending the same generosity to future researchers aligns with the very nature of academia as a community devoted to teaching and learning” (p. 49).

Acronyms

CoARA - The Coalition for Advancing Research Assessment

DORA - Digital Operational Resilience Act

AJOL - African Journals Online

SciELO - Scientific Electronic Library Online

NSF - National Science Foundation

Conflict of Interest

No conflicts of interest were identified.

Other

This review represents the opinions of the authors and does not represent the position of Future of Research Evaluation and e-Scholarship (FORCE11) as an organization.

Competing interests

The authors declare that they have no competing interests.

Use of Artificial Intelligence (AI)

The authors declare that they did not use generative AI to come up with new ideas for their review.