Breaking Down Accuracy with Subspace Optimization

This paper introduces a Python toolkit to break down accuracy of classification models, by identifying high-accuracy portions of the test dataset and thus facilitating a deeper understanding of where the model performs best and where it falls short. Given a dataset, a classification model and an accuracy thresh- old, our system returns a range query that selects the largest sub-space of the test dataset on which the classifier achieves higher accuracy. The toolkit allows users to interactively explore such sub-space, by adjusting the returned ranges with graphical elements and observing the change in the model’s accuracy and data distributions. Ranges can be manually initialized to highlight the strengths and weaknesses of the model in different scenarios. The core of our method consists of a mixed-integer optimization algorithm. Demonstration on real-world datasets and a selection of models show that our toolkit can serve as an effective way to understand performance across different data segments.