High resolution satellite imagery orientation accuracy assessment by leave-one-out method: accuracy index selection and accuracy uncertainty

The Leave-one-out cross-validation (LOOCV) was recently applied to the evaluation of High Resolution Satellite Imagery orientation accuracy and it has proven to be an effective method alternative with respect to the most common Hold-out-validation (HOV), in which ground points are split into two sets, Ground Control Points used for the orientation model estimation and Check Points used for the model accuracy assessment. On the contrary, the LOOCV applied to HRSI implies the iterative application of the orientationmodel using all the known ground points as GCPs except one, different in each iteration, used as a CP. In every iteration the residual between imagery derived coordinates with respect to CP coordinates (prediction error of the model on CP coordinates) is calculated; the overall spatial accuracy achievable from the oriented image may be estimated by computing the usual RMSE or, better, a robust accuracy index like the mAD (median Absolute Deviation) of prediction errors on all the iterations. In this way it is possible to overcome some drawbacks of the HOV: LOOCVis a reliable and robustmethod, not dependent on a particular set of CPs and on possible outliers, and it allows us to use each known ground point both as a GCP and as a CP, capitalising all the available ground information. This is a crucial problem in current situations, when the number of GCPs to be collected must be reduced as much as possible for obvious budget problems. The fundamentalmatter to deal with was to assess howwell LOOCVindexes (mADand RMSE) are able to represent the overall accuracy, that is howmuch they are stable and close to the corresponding HOV RMSE assumed as reference. Anyway, in the first tests the indexes comparison was performed in a qualitative way, neglecting their uncertainty. In this work the analysis has been refined on the basis of Monte Carlo simulations, starting from the actual accuracy of ground points and images coordinates, estimating the desired accuracy indexes (e.g. mAD and RMSE) in several trials, computing their uncertainty (standard deviation) and accounting for them in the comparison. Tests were performed on a QuickBird Basic image implementing an ad hoc procedure within the SISAR software developed by the Geodesy and Geomatics Team at the Sapienza University of Rome. The LOOCV method with accuracy evaluated by mAD seemed promising and useful for practical cases