Improving reliability in deep learning: exploring generalization bounds, noisy label handling, and loss surface characterization

This dissertation investigates the theory of generalization and robustness in deep learning. Through diverse research works, the thesis provides valuable insights and advancements towards building more reliable systems. We focus on handling noisy labels, deriving generalization bounds for clustering, enhancing interpretability, and characterizing the topology of the loss landscape. These findings contribute to the broader field of deep learning, advancing the development of effective and reliable machine learning systems. After the introduction provided in Chapter 1, in the second chapter we tackle the challenge of noisy labels in classification by leveraging inter-rater agreement and estimating the noise distribution, thereby improving model performance and robustness. In the third chapter, we establish generalization bounds for projective clustering, presenting near-optimal results for subspace clustering. Chapter 4 introduces a novel artificial neuron that enhances interpretability while retaining the representation power and performance of a standard neural network. Indeed, we prove the universal approximation theorem for specialized versions of the artificial neuron. In Chapter 5, we characterise the topological complexity loss surfaces using Betti Numbers. Understanding the topology of loss surfaces is crucial for studying generalization and robustness in deep learning models. The last chapter summarizes the key findings and contributions, also mentioning possible future directions. Collectively, this research work contributes to understanding generalization and robustness in deep learning, advancing the field and enabling the development of more reliable models.