Comparing Penalized Regression Analysis of Binary Logistic Regression Model with Multicollinearity Using Log Loss

Abstract

In this study compares the performance of the ridge, lasso, and elastic net approaches for controlling multicollinearity issues among independent variables in binary logistic regression analysis. A binary logistic model with eight independent variables (P=8) and a high degree of multicollinearity (ρ=0.99) at various sample sizes (20,50,100,200, and 300) is applied in data simulation. The best approach is found using the minimum logarithmic loss (Log Loss) and Akaike's information criterion (AIC) values. According to the study results, the lasso method consistently offers the lowest Log Loss and AIC, making it the best model. For every sample size tested, ridge method seems to be the least successful of the three models, whereas elastic net is a powerful substitute for lasso method.