We introduce a novel computational unit for neural net- works that features multiple biases, challenging the tradi- tional perceptron structure. This unit emphasizes the impor- tance of preserving uncorrupted information as it is passed from one unit to the next, applying activation functions later in the process with specialized biases for each unit. Through both empirical and theoretical analyses, we show that by focusing on increasing biases rather than weights, there is potential for significant enhancement in a neural network model’s performance. This approach offers an al- ternative perspective on optimizing information flow within neural networks.
Increasing biases can be more efficient than increasing weights / Metta, C.; Fantozzi, M.; Papini, A.; Amato, G.; Bergamaschi, M.; Galfre, S. G.; Marchetti, A.; Veglio, M.; Parton, M.; Morandin, F.. - ELETTRONICO. - 28:(2024), pp. 2798-2807. (Intervento presentato al convegno 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) tenutosi a Waikoloa, HI, USA nel 03-08 January 2024) [10.1109/WACV57701.2024.00279].
Increasing biases can be more efficient than increasing weights
Metta C.;Fantozzi M.;Amato G.;Marchetti A.;Veglio M.;Parton M.;Morandin F.
2024-01-01
Abstract
We introduce a novel computational unit for neural net- works that features multiple biases, challenging the tradi- tional perceptron structure. This unit emphasizes the impor- tance of preserving uncorrupted information as it is passed from one unit to the next, applying activation functions later in the process with specialized biases for each unit. Through both empirical and theoretical analyses, we show that by focusing on increasing biases rather than weights, there is potential for significant enhancement in a neural network model’s performance. This approach offers an al- ternative perspective on optimizing information flow within neural networks.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.