Learning Variational Models via Bilevel Optimization and Unfolded Algorithms

Variational models are a classical tool to solve inverse problems in a multitude of contexts. In contrast to other modern methodologies, they lay their foundations on an established theoretical background. The advantage offered by their interpretability, combined with a less intense requirement in terms of computational resources makes the study and use of them still relevant to this day. To improve the performance of these models, we address one of their main weaknesses: parameters selection. These models are defined through an energy function which itself is characterized by one or more variables. The absence of broad rules to set these variables often leads to a tedious, and perhaps time consuming, empirical search for a good configuration. Borrowing ideas from the machine learning realm to automate this process, we effectively train the energy functional with the goal of finding a more than satisfying parameter setup. We study a general formulation of a bilevel optimization problem to carry out this task in a variety of imaging applications. In detail, we develop an iterative general purpose inexact forward-backward algorithm which is able to converge to a stationary point of the bilevel problem. The experiments show that the algorithm is able to find more than valid solutions in a small amount of time. Although these results are promising, there are still limitations to the approach. To circumvent some of these, we also relax the lower level minimization problem by replacing it with the unfolding of an iterative optimization algorithm. The number of iterations that the chosen scheme performs is fixed a priori, so that the whole lower level resembles a very basic neural network. The training of this algorithm unfolding then aims at finding the best parameters (including some of the optimization algorithm itself!) to make the most out of the given iterations. We tested these ideas on different imaging problems, with surprising results, especially when compared to deep learning methods.