Explaining the Inexplicable: An Explainable AI Approach to Boost Knowledge Transfer

Deep learning has revolutionized performance across diverse applications but is often criticized for its opaque nature and computational complexity, complicating efforts to understand and trust model predictions. Explainability techniques have emerged to address these issues, enhancing model transparency and user trust. Simultaneously, efficiency techniques such as Knowledge Distillation offer solutions to streamline complex (teacher) models into more efficient (student) ones while preserving their predictive capabilities. Building on those research efforts, this paper presents a general framework that is able to improve well-known distillation techniques through explainability methods boosting both model interpretability and accuracy of shallow student architectures. More in detail, by incorporating a novel loss function that aligns the explainability maps of teacher and student models, our method refines the distillation process, leading to more accurate and interpretable predictions. Our approach not only improves the performance of distilled models but also demonstrates the effective integration of explainable insights into distillation frameworks, as validated by multiple experiments on a publicly available benchmark dataset.