Assessing the reliability of species distribution projections in climate change research

Aim: Forecasting changes in species distribution under future scenarios is one of the most prolific areas of application for species distribution models (SDMs). However, no consensus yet exists on the reliability of such models for drawing conclusions on species’ distribution response to changing climate. In this study, we provide an overview of common modelling practices in the field and assess the reliability of model predictions using a virtual species approach. Location: Global. Methods: We first review papers published between 2015 and 2019. Then, we use a virtual species approach and three commonly applied SDM algorithms (GLM, MaxEnt and random forest) to assess the estimated and actual predictive performance of models parameterized with different modelling settings and violations of modelling assumptions. Results: Most SDM papers relied on single models (65%) and small samples (N < 50, 62%), used presence-only data (85%), binarized models' output (74%) and used a split-sample validation (94%). Our simulation reveals that the split-sample validation tends to be over-optimistic compared to the real performance, whereas spatial block validation provides a more honest estimate, except when datasets are environmentally biased. The binarization of predicted probabilities of presence reduces models’ predictive ability considerably. Sample size is one of the main predictors of the real model accuracy, but has little influence on estimated accuracy. Finally, the inclusion of ecologically irrelevant predictors and the violation of modelling assumptions increases estimated accuracy but decreases real accuracy of model projections, leading to biased estimates of range contraction and expansion. Main conclusions: Our ability to predict future species distribution is low on average, particularly when models’ predictions are binarized. A robust validation by spatially independent samples is required, but does not rule out inflation of model accuracy by assumption violation. Our findings call for caution in the application and interpretation of SDM projections under different climates.