Avalilação PREreview de Investing in Open Science: Key Considerations for Funders

This review is the result of a virtual, collaborative live review discussion organized and hosted by PREreview and the PREreview Club at Future of Research Communication and e-Scholarship (FORCE11) on July 22, 2026. The discussion was joined by 7 people: a facilitator and 6 additional live and asynchronous review participants. The authors of this review have dedicated additional asynchronous time over the course of two weeks to help compose this final report using the notes from the live and asynchronous reviews. Morufu Olalekan Raimi also participated in the discussion. We thank all participants who contributed to the discussion and made it possible for us to provide feedback on this preprint.

Summary

This manuscript presents a practice-based account of the Aligning Science Across Parkinson’s (ASAP) Initiative’s experience implementing a comprehensive Open Science policy. Rather than evaluating Open Science policies experimentally, the paper describes the practical, organizational, financial, and logistical considerations involved in designing, implementing, and enforcing such a policy. Organized around the five core requirements of the ASAP Open Science Policy, the manuscript explains the barriers encountered, the tools and services adopted, the staffing and infrastructure required, and the policy adaptations made as implementation progressed. In doing so, it provides a detailed case study that may help other funders better understand the resources and institutional capacity needed to support Open Science in practice.

Overall, we found the manuscript to be timely, informative, and highly relevant for research funders and organizations developing Open Science policies. We particularly appreciated its transparent presentation of implementation costs, practical challenges, and lessons learned, as well as its detailed descriptions of the infrastructure, services, and workflows that supported policy implementation. One of the manuscript’s most valuable contributions is demonstrating that adopting an Open Science policy involves much more than publishing articles in open access: successful implementation also requires sustained investment in training, specialized staff, compliance monitoring, infrastructure, incentives, and ongoing organizational support. The practical orientation of the paper and the openness with which the authors share their experience make it a useful resource for organizations considering similar initiatives.

List of major concerns and feedback

While we found the manuscript to be a valuable and timely contribution, we identified several issues that we believe should be addressed before publication. Most of these concerns do not challenge the value of the work itself, but rather relate to methodological transparency, clarity of scope, and the interpretation of some of the findings. Addressing these points would strengthen the manuscript and increase its usefulness for a broader range of funders, institutions, and research communities.

1. The manuscript would benefit from greater methodological transparency: One of our main concerns relates to the methodological description supporting the analyses presented throughout the manuscript. Although the paper provides valuable cost estimates and openly shares datasets through Zenodo, it is often unclear how different costs were collected, verified, or estimated. Some values appear to represent actual ASAP expenditures, whereas others seem to be vendor estimates or broad cost ranges, but the criteria used to distinguish these categories are not fully explained.

Similarly, additional methodological detail would help readers understand how publications were selected, how missing data were handled, how compliance measures were generated from DataSeer reports, and how the analyses underlying the figures and tables were performed.

Suggestion: We encourage the authors to expand the methodological description both in the manuscript and, where appropriate, in the accompanying documentation. Providing clearer information on data collection, inclusion criteria, analytical procedures, and the role of DataSeer’s AI-assisted compliance assessment would substantially improve transparency and make it easier for other organizations to reproduce similar analyses.

1. The scope of the manuscript and the generalizability of its conclusions should be clarified: The manuscript provides a rich description of ASAP’s implementation experience, but its overall framing sometimes makes it difficult to distinguish between lessons derived from a single institutional case and conclusions that may be applicable to research funders more broadly. Several reviewers noted that the manuscript reads more like an implementation report, practice-based case study, or program evaluation than a conventional research article, yet this distinction is not explicitly stated.

Suggestion: We suggest that the authors clearly define the article type and explicitly acknowledge that the findings arise from one well-resourced funding initiative operating within a specific research context. Discussing the absence of comparison groups, the context in which ASAP operates, and the potential limits to generalizability would help readers better interpret which recommendations are broadly transferable and which are more context dependent.

1. The discussion of Open Access publishing models would be strengthened by additional balance and contextualization: Several reviewers appreciated that the manuscript recognizes multiple routes to Open Access, including self-archiving, Diamond (also referred to as Platinum) Open Access journals, and preprint-based dissemination, rather than presenting APC-funded publishing as the only option. At the same time, reviewers expressed concern that some statements—particularly regarding Diamond Open Access—may be interpreted more broadly than the evidence presented supports. In particular, the conclusion that Diamond Open Access is “not yet an ideal route” because only a relatively small proportion of journals comply with specific cOAlition S standards may inadvertently conflate compliance with one policy framework with the broader viability or quality of Diamond Open Access as a publishing model.

Suggestion: We encourage the authors to further contextualize this discussion by making clearer that different Open Access approaches and publishing pathways operate under different policy environments and infrastructures. A more nuanced presentation of the strengths and limitations of APC-funded publishing alongside alternative models would strengthen the manuscript and help readers avoid drawing overly broad conclusions from one particular implementation context.

1. Figure 1 and its supporting data could be improved by additional verification and clearer documentation. Several reviewers identified concerns regarding Figure 1 and the dataset deposited in Zenodo that supports it. Attempts to reproduce the figure yielded results that differed substantially from those presented in the manuscript, and reviewers identified inconsistencies between the deposited files, the README documentation, and the figure itself. Additional questions were also raised regarding journal selection criteria, the source and timing of impact factor data, and the interpretation of the relationship between journal impact factor and APCs.

Suggestion: Given the importance of this figure to the discussion of publishing costs, we strongly encourage the authors to verify the underlying data, review the figure generation process, and ensure consistency between the deposited datasets, documentation, and published visualizations. Providing additional methodological detail and reconsidering the presentation of Figure 1 would improve both reproducibility and interpretation of the results.

1. The manuscript would be enhanced by a more explicit discussion of limitations and study context: Although the manuscript openly discusses many implementation challenges, reviewers felt that it would benefit from a dedicated limitations section. In particular, the reported costs largely reflect an early implementation phase and may change over time as workflows mature, institutional learning accumulates, and supporting infrastructure becomes more established. Likewise, the manuscript focuses on a single funding initiative without explicitly discussing how organizational context, available resources, or disciplinary differences may influence implementation elsewhere.

Suggestion: We encourage the authors to include a more explicit discussion of these limitations, including the potential effects of organizational learning on future costs, the context-specific nature of the ASAP model, and the implications of drawing conclusions from a single implementation experience. Such additions would provide readers with a clearer understanding of the scope and applicability of the findings.

List of minor concerns and feedback

In addition to the major points above, reviewers identified several smaller issues that, while not affecting the overall contribution of the manuscript, could improve its clarity, readability, and usefulness for readers.

1. The organization of the manuscript could be more balanced: Several reviewers noted that the discussion is heavily weighted toward the first policy requirement, while the remaining four requirements receive considerably less attention. As a result, the manuscript occasionally feels uneven in its structure.

The authors may wish to consider organizing the discussion around broader implementation themes—such as implementation barriers, costs, infrastructure, training, incentives, and lessons learned—while using the five policy requirements as examples within those broader sections. This approach could improve the overall balance and readability of the manuscript.

1. Several tables and figures would benefit from improvements in clarity and presentation: The tables constitute one of the strongest aspects of the manuscript because they make implementation costs and operational considerations highly visible. Nevertheless, reviewers identified several opportunities to improve their readability. These include spelling out acronyms so that tables can be understood independently, explaining why the header row is repeated in Table 1, improving the readability of underlined text in printed versions, and clarifying the meaning of the categories used in Table 5 (e.g., “Publication Version = 1” versus “Publication Version >1”).

Reviewers also suggested improving the presentation of Figure 2 by providing a higher-resolution image or dividing it into smaller panels to facilitate interpretation.

1. Additional contextual information would help readers interpret the findings: Several reviewers suggested that the manuscript could provide more background information about the ASAP initiative itself, including its research context, funding context, and scope of activities. Additional context would help readers better understand how the implementation experience described in the manuscript relates to other funding organizations and research settings.

2. Some ethical and transparency considerations could be described more explicitly: Although the manuscript appropriately discloses the authors’ institutional affiliations and openly shares the underlying datasets, reviewers suggested providing additional information regarding the handling of compliance data, including whether individual-level information was anonymized, whether ethics review was required or considered exempt, and how potential conflicts of interest were managed in what is essentially an internal evaluation.

Including these clarifications would further reinforce the transparency standards that the manuscript itself promotes.

1. The discussion of licensing could benefit from additional clarification: The manuscript indicates that publications may be released under either a CC BY 4.0 or CC0 license. Some reviewers suggested that this point could be clarified further, particularly regarding the circumstances under which CC0 would be appropriate for scholarly publications and how it differs from CC BY in this context.

Overall Assessment

Overall, we found this manuscript to be a valuable, timely, and highly practical contribution to the growing literature on Open Science implementation. One of its greatest strengths is its transparency in documenting the organizational, financial, and human resources required to translate Open Science principles into everyday research practice. By openly sharing implementation experiences, operational challenges, cost estimates, and supporting resources, the authors provide insights that are rarely available in the literature and that will likely be useful to funding organizations and others developing or refining Open Science policies.

Our comments are intended to strengthen an already valuable contribution. We believe that greater methodological transparency, a clearer definition of the manuscript’s scope and article type, a more explicit discussion of contextual limitations, and additional nuance in the discussion of Open Access publishing models would enhance both the rigor and the broader relevance across research communities. Addressing these points would help readers better understand which aspects of the ASAP experience are context-specific and which may be applicable to other organizations working to advance Open Science. We hope our suggestions are received in the collaborative spirit in which they are offered and believe that, with these revisions, the manuscript has the potential to become an important reference for funders and research organizations seeking to implement ambitious and sustainable Open Science policies.

Concluding remarks

We thank the authors of the preprint for posting their work openly for feedback. We also thank all participants of the Live Review call and asynchronous discussion for their time and for engaging in the lively discussion that generated this review.

This review represents the opinions of the authors and does not represent the position of Future of Research Communication and e-Scholarship (FORCE11) as an organization.

Competing interests

The authors declare that they have no competing interests.

Use of Artificial Intelligence (AI)

The authors declare that they did not use generative AI to come up with new ideas for their review.