Multicollinearity

In regression, multicollinearity refers to predictors that are correlated with other predictors.  Moderate multicollinearity may not be problematic.  However, severe multicollinearity is problematic because it can increase the variance of the regression coefficients, making them unstable and difficult to interpret.

To measure multicollinearity, you can examine the correlation structure of the predictor variables. You can also review the variance inflation factor (VIF), which measures how much the variance of an estimated regression coefficient increases if your predictors are correlated. If the VIF = 1, there is no multicollinearity but if the VIF is > 1, predictors may be moderately correlated.  When the VIF is 5 - 10, the regression coefficients are poorly estimated.

Possible solutions to severe multicollinearity:

·    Remove highly correlated predictors from the model.  Because they supply redundant information, removing them often does not drastically reduce the R2.  Consider using stepwise regression, best subsets regression, or specialized knowledge of the data set to remove these variables.

·    Use Partial Least Squares Regression (PLS) or Principal Components Analysis.  These methods reduce the number of predictors to a smaller set of uncorrelated components.

For example, a toy manufacturer wants to predict customer satisfaction from survey results and initially includes "strength" and "lack of breakage" as predictor variables in the regression model.  The investigator finds that these two variables are strongly negatively correlated and have a VIF greater than 5.  At this point, the investigator could try removing either variable or use PLS or Principal Components Analysis to use these related variables to create a "durability" component.