PREreview of Overconfidence undermines global wildlife abundance trends

This review was made as part of a community of practice supported by ASAPbio and BIOS². One of the authors, GH, is a signatory of Publish Your Reviews, and has committed to publish her peer reviews alongside the preprint version of an article. For more information, see publishyourreviews.org.

Authors’ background:

Open science, open data, network ecology, plant ecology, conservation biology, taxonomy, ecological modelling, plant biology, seed science and technology, agronomy, urban evolutionary biology, biodiversity change, indicators, population dynamics, island biogeography, computational and quantitative ecology, prediction, community ecology, eDNA, metacommunity.

—-

Summary:

The authors propose a statistical framework that addresses an important need to incorporate the correlations between species in their estimation of biodiversity change trends. The authors develop a correlated effect model that can incorporate phylogenetic, spatial, and temporal correlations into the estimation of wildlife abundance trends, and demonstrate it on 10 widely-used biodiversity datasets. Accounting for the non-independence between species alters estimated trends and can, in some cases, reverse their direction. The correlated effect model also forecasts wildlife abundance trends with greater accuracy. Importantly, their model reveals that uncertainty is underestimated in biodiversity change models, and the authors highlight the importance of transparency about this uncertainty.

The result is an impressive showcase of their proposed framework to estimate and forecast wildlife abundance trends, the results of which build a convincing argument for accounting for the non-independence between species when modelling biodiversity change. The paper offers a concise and clear explanation of the rationale and application of this framework, with greater detail in the supplementary information for those who want to delve deeper into the model.

Big-picture comments

Good/Excellent things

The paper is well sold and having a primarily results-focused manuscript really helps to convey how this new modelling approach fills existing gaps in a high-level manner but still makes the nitty gritty details available within the supplementary material. Well done! The study itself is extremely relevant for biodiversity change assessments because it addresses something that has until now mostly been theoretically discussed, and not yet implemented for wildlife abundance trends.

The figures are visually pleasing and well-made! They provide good support for the text and make comprehension easier. The style of writing is accessible and consistent, which is helpful to understand some of the more complex ideas in their proposed framework.

The use of so many datasets, from so many different sources, with so much information lends a lot of credibility to the conclusions made by the authors and one can only imagine the work to put everything in the same format! It is also so good to see such a great demonstration of the benefits (and power) of open data and collaboration. Kudos for acknowledging all the data providers.

Things to be improved

The “Implications for biodiversity change” section is important, and the method that the authors propose should be a useful tool for gleaning more accurate trend estimates from imperfect data, which is an invaluable contribution in a context where such estimates inform policy. 

That said, we feel that this section was somewhat underdeveloped. While we would understand if the authors feel that a discussion of policy implications is beyond the scope of the manuscript, we suspect that, after conducting such an in-depth exploration of sources of uncertainty in biodiversity data, the authors could provide more insights into the technical/methodological implications of their findings. For example, the analysis relied on a trimmed dataset that selected the best available data. Would the authors propose taking a similar approach when analysing biodiversity data in a policy context? Should we use only data for species for which we have high-quality data and phylogenetic information? 

Furthermore, this section could also be expanded to discuss the wide confidence intervals of the model. The variance itself is not a bad thing per se but may cause some uncertainty amongst end-users, and raises methodological questions as to how we should interpret and account for uncertainty (both from the model as well as underlying) when making policy decisions.

Finally, are there outstanding questions that require further study? Given that the study relied upon real-world datasets, we don’t know the “truth” that underlies the abundance estimates used in the analysis. Could a simulation study be a good complement to the analysis presented here? 

Please consider the above to be suggestions, as the authors have much more knowledge and insight about the analysis than we do. However, we encourage the authors to share their thoughts on the importance and application of the model.

Small-picture things

Writing

We feel like transitions between paragraphs could maybe be a bit smoother. Adding a last sentence that allows the transition to the next paragraph could add to the flow of the text. More specifically, the transition between the first and the second paragraph could be better framed. In the closing out of the first paragraph, it feels like the authors were cautioning the reader as to the quality of datasets but then it jumps to talking about methodology. It might be worth rephrasing the closing sentences of the first paragraph to make it clear that it is the ‘artefacts’ of spatiotemporal data that make modelling challenging and not that the data quality is poor per se.

Figures

As for Figure 2, it could feel a bit confusing to use blue for decreasing trends and red for increasing trends. It also did not convince some reviewers that the models are so different, with this figure. Maybe it’s just not communicating what it should communicate, but for us, it seems like the difference between the new method and previous models is not so substantial, since most of the time it shows a non-significant decrease or increase (and so do most of the other models), and when it does show significant increase or decrease, so do the other models. Additionally, the credible intervals very frequently include the trends from other models, which makes us think that they are not different, but the interpretation of each model can be (in terms of uncertainty). Also, the axes are not aligned with each other. Since we are talking about trends and not about each time series and species (that would likely have different limits), the x-axis could have the same limits for all figures.

In the global-scale column of Figure 3, it is difficult to distinguish between site-level trends (grey line) and confidence intervals (shading). Perhaps using different colours would make the distinction more clear.

The overall message of Figure 4 could be clarified. The shading on the map panel in this figure could also be reworked, as it could give the false impression that rates of change are higher in the oceans than on the continent. We also propose to replace Figure 4 with Figure S1.

Discussion

The “Implications for biodiversity change” section is probably the most important one. It needs to be impactful and really make us know how dangerous it is to misestimate population trends. The method presented in the paper is important because it added uncertainty where we had less uncertainty - which is a good thing! I think authors could drive away from contrasting their model to the other ones, and instead build on how they can complement each other. Is it actually dangerous to consider a species' population as decreasing when it's actually “non-significantly decreasing”? What is the actual, policy-wise difference between “non-significantly increasing” and “non-significantly decreasing”? Would policies based on this trend differ between countries, species, and ecosystems? At the same time, it is possible to see how some of the phrasings in this section can be dangerous, politically: to claim that the “validity of the existing methods for describing global or regional abundance change where datasets are spatially, temporally, or phylogenetically non-independent” should be seriously questioned can be easily used to nourish anti-conservation movements.

Closing remarks

That's a very important paper, and the writing and illustrations are really remarkable. It is a relevant gap in the scientific reasoning the fact that autocorrelation was not being considered in population trend models. To develop such a method is alone a praiseworthy deed, and even more by using 10 datasets from different sources. The description of uncertainty is always a much-needed result in ecology and biodiversity conservation, and this method adds to that. This paper should definitely be published.

However, it lacks some political and social considerations that are the spine of biodiversity conservation science. It seems to us that 1. These results might have different meanings in different social and political contexts; and 2. It is dangerous to question previous methods and results in this case, as they might significantly change conservation policies worldwide. 

Additionally, we believe some of the Supplementary Materials should be on the main text if possible. We understand the preprint might be meant to be published as a short format manuscript in a journal, but considering the preprint as a final product, we think that a summary table describing the types of models and some explanations about the “advanced statistics” terminology would be helpful to the understanding of the paper. Finally, we suggest that the transitions between the paragraphs to be reviewed, as sometimes they “jump” between subjects without connecting them very clearly.

Overall, we are very impressed with the paper and we really think it’s an important contribution to science.

Competing interests

The author declares that they have no competing interests.