A Bayesian Approach to the Forward Search

Chapter 1 introduces the basics of inferential statistics, starting from the frequentist approach, recalling the method of maximum likelihood estimation that will play a very important role in the understanding of the Bayesian approach. Some derivations of maximum likelihood functions are then presented, that will accompany us throughout the thesis, turning into their Bayesian equivalents. In the same chapter we then introduce the theoretical foundations of the Bayesian framework showing the derivation of the normal model and the linear regression model in the Bayesian framework. Chapter 2 starts introducing some mathematical background on the linear regression model and exploring the FS framework applied to the linear regression, with the monitoring of the minimum deletion residual, the leverage and Mahalanobis distance trajectories. Then an example of this method applied to the Gross Domestic Product of 27 OECD Countries is discussed, showing how we can detect groups of outliers and how can be assessed, given an algorithm of variable selection, which is the best linear model that describes the data. The foundations of the Bayesian regression in the case of informative and uninformative priors and the normal-inverse gamma model of beta and sigma^2 of the parameters distribution are considered, altogether with the Gibbs sampling algorithm as the main algorithm to draw simulations. The application of the Bayesian framework to the FS is introduced and variable selection or BMS is applied, outlining all the similarities and differences between the different approaches. Finally, an example of credit risk estimation for bankruptcy prediction in manufacturing firms is shown with a discriminant approach based on a generalized linear model, to see if including prior information could help to best discriminate bankrupted firms form the healthy ones. In chapter 3 we discuss the technique of BMA (BMA) and we show how it is possible to robustify this technique embedding it in the FS context. In this chapter we recall the data abut GDP of 27 OECD countries seen in chapter 2 an we highlight the peculiarity of BMA. More precisely we clearly show that the application of standard BMA can be highly misleading to suggest what are the most important variables. Only the monitoring of the trajectories of the posterior probabilities of inclusion of the variables enables us to understand, free of masking and swamping effect, the real importance of the different regressors and the effect that the most remote observations exert on the different model specifications.