Linear Regression¶

Linear regression is one of the most popular regression algorithms and produces good predictions for well-prepared data. Its optimization function computes coefficients to express a response column as a linear relationship of its predictors.

You must verify the Gauss-Markov assumptions when using linear regression algorithms:

• Linearity : the parameters we are estimating using the OLS method must be linear.
• Non-Collinearity : the regressors being calculated aren’t perfectly correlated with each other.
• Exogeneity : the regressors aren’t correlated with the error term.
• Homoscedasticity : no matter what the values of our regressors might be, the error of the variance is constant.

To create a good linear regression model, it's important to:

• Impute missing values
• Encode categorical features (linear regression only accepts numerical variables)
• Compute the correlation matrix to retrieve highly-correlated predictors
• Decompose the data (optional)
• Normalize the data (optional, but recommended)

Example without decomposition¶

Let's use the 'africa_education' dataset to compute a linear regression model of students' performance in school.

First, let's look for missing values.

We'll simply drop the missing values to avoid adding bias to the data.

We need to encode the categorical columns to dummies to retain linearity.