Transformative approaches for deep Learning in resource-constrained scenarios

Latest advancements in Artificial Intelligence, and in particular in Deep Learning, have catalyzed groundbreaking progress across diverse applications such as Computer Vision, Natural Language Processing, and content generation. However, the significant computational demands inherent in training and executing powerful Deep Learning models have hindered widespread adoption of these techniques in certain application contests. One area poised to benefit greatly from Deep Learning, especially applied to computer vision, is the automotive sector, particularly in the development of driver assistance systems. In this context, minimizing inference costs is a priority in order to enable deployment on the low-power embedded devices found in vehicles. Conversely, the costs and complexities associated with training phase are sometimes substantial, exemplified by recent transformer-based models for natural language processing and image synthesis models utilizing the Denoising Diffusion Probabilistic paradigm. This thesis addresses two primary objectives: (i) proposing low-computational-cost solutions for computer vision applications in automotive settings and (ii) presenting innovative approaches to formulating efficient Deep Learning models using lossy compression techniques. To achieve the former goal, this study develops two models for Driver Monitoring Systems and Advanced Driving Assistance Systems, employing a Multi-Task Learning approach. This choice enable significant computational savings by sharing a substantial portion of the neural architecture across different tasks. For the latter objective, this thesis introduces an approximation of the transformer attention layer leveraging the Discrete Cosine Transform. Additionally, it proposes a strategy for incorporating Vector Quantization-based compression techniques into the image generation process using Diffusion Models. Through experimental analyses and quantitative evaluations, this thesis demonstrates the effectiveness of the proposed methods in reducing the complexity and computational costs in the concerned contexts.