Optimized training and scalable implementation of Conditional Deep Neural Networks with early exits for Fog-supported IoT applications

The incoming IoT big data era requires efficient and resource-constrained mining of large sets of distributed data. This paper explores a possible approach to this end, combining the two emerging paradigms of Conditional Neural Networks with early exits and Fog Computing. Apart from describing the general framework, we provide four specific contributions. First, after reviewing the basic architectures of CDNNs with early exits and characterizing their computational capacity, we consider three basic algorithms for their supervised training (namely the End-to-End, Layer-Wise and Classifier-Wise training algorithms), and, then, formally characterize and compare the resulting tradeoffs in a Fog-supported implementation. Second, after presenting a reference architecture for the local classifiers equipping the considered CDNNs, we develop an optimized framework for the parallel and distributed setting of their decision thresholds. Third, we propose a greedy algorithm for placing the early exits efficiently on the considered CDNNs and prove its linear scaling complexity. Fourth, we analytically characterize in closed-form and analyze the energy performance of the optimal CDNN-onto-Fog mapping. Finally, extensive numerical tests are presented, in order to test and compare the energy-vs.-implementation complexity-vs.-accuracy performance of the resulting optimized CDNN-over-Fog platforms under the IoT-oriented SVHN and FER-2013 datasets.