PREreview of Systemic racial disparities in funding rates at the National Science Foundation

Overall Summary:

This preprint analyzes National Science Foundation data on funding rates, award types, and proposal ratings from 1996 to 2019 to investigate racial disparities in funding. The preprint finds that: 

• White investigators are funded at higher rates;
• Funding rates for White PIs increased over the time period in question;
• Disparities occur across all directorates at NSF; and
• Disparities are greatest for Research Proposals in particular.

The introduction is succinct and concise, and includes the important comparison with the data about funding disparities at the National Institutes of Health.

The results section presents the data in a very clear manner. The figures are very well-made and clearly illustrate the points being made.

The Supplementary data section is impressive and a really useful resource for those wishing to use this work looking at various subsets of the issue and at particular at the level of the individual directorates. In addition, the authors give a very clear and very helpful description of NSF processes, of the limitations to the data and changes to its collection over time, the suggestions for future work, and the caveats to the terms used and applied in data collection efforts. Overall this is a very useful resource for the community, and beyond the results themselves this was an excellent use of the opportunity to share as much information as possible to the community.

Major Comments:

I have no major critiques of the paper, other than this is a well-written and useful contribution to the analysis of racial disparities in federal funding.

Minor comments:

The authors talk about surplus/unfunded awards in a way that’s very helpful to understand (Figure 2, Figure 4). However the numbers of applications and awards vary drastically by demographic, as shown in Figure 2 - for example, there are more proposals submitted from PIs who are White than all other groups combined, and there are twice as many proposals from White PIs as from Asian PIs, who in turn outnumber Hispanic/Latino proposals 4-fold, etc. This means that a simple summation of the surplus or unfunded awards leads to a number that isn’t zero (+364, by my calculation) - when using the overall funding rate stated. Put another way, one could perhaps calculate a new funding rate where all groups are funded at an equal rate, to give no net funded . It could be useful to know what that number is, because it seems like it might actually give a higher overall number for funding, which could be important to know for discussions about how increased focus on equitable funding could lead to an increased number of proposals being funded. This could then also be applied at the directorate level. It may not be useful to calculate this number, however, if there are other caveats to consider, and others may have critiques of this proposed analysis that I have not considered or fully appreciate at this time.

I would be interested in seeing more discussion about the early vs late career stage data in Figure S7, especially if the authors can comment on the intersection between this and the other demographic data. The funding disparity is stark and retention within faculty roles may be a key part of the wider story and while the authors do mention this in the main text, it is dealt with very briefly.

The nature of the PDF being so long makes it a little unwieldy to read it times (although I appreciate there is little the authors can do, and it is very well laid-out and easy to read generally). But some things that could help would be: a section where abbreviations are defined to allow easy reference; and maybe page numbers for the figures just to make jumping around easier. Otherwise the organization and layout of the data is very thoughtfully done, given how much there is to go through, and the authors did a wonderful job making this so readable. I appreciate that the main figures are in the main text, which for most readers is more important.

It may be helpful to readers to restructure the discussion, and Figure S1, to better clarify that Research Proposals + Non-Research Proposals = Competitive Proposals - this may not be immediately apparent, and e.g. Figure S1 could suggest that there are 3 differing types of proposal. If could also be that a small figure showing this would help guide the reader unfamiliar with NSF terminology. I think this would also be helpful given the discussion in Figure S4, that Competitive proposals are not “all” proposals, but exclude a number of application types, the largest likely being the Graduate Research Fellowship Program. Showing all the various kinds of proposals and how they relate to what you discuss in a figure would be very helpful to guide the reader.

Conflicts:

I have no conflicts to report; I do not know the authors personally and have not been involved in any way with the work. I do not stand to gain or suffer financially or otherwise from this publication.

License:

This review is published under a CC-BY license.