You are viewing a public preview
Sign in to save your progress, do assignments, join discussions and get feedback.
After this unit you will: explain what Linear Regression does, and distinguish simple regression from multiple regression.
y = ω₁x₁ + ω₂x₂ + … + ωₚxₚ + ω₀ + ε
By the end of this lesson you can:
LinearRegressionKey idea: Linear Regression assumes a roughly straight-line relationship between the features and a numeric target, and it learns the line (or flat surface) that keeps the average squared prediction error as small as possible.
The lecture opens with a scatter plot: salary on the vertical axis, years of experience on the horizontal axis. The points follow an upward diagonal trend, and a green straight line runs through them. That line is what the model learns. One red point sits above the general trend; no straight line passes through every point, so the model aims for the line that fits the overall pattern best.
Linear Regression is a good first algorithm for Module 2 because the idea is simple: as experience goes up, salary tends to go up in a consistent, linear way. Because the target (salary) is a continuous number and not a category, this is a regression problem.
The learned line is the classic equation y = m*x + b:
| Symbol | Meaning in the example |
|---|---|
| y | predicted salary |
| x | years of experience |
| m (slope) | how much salary increases for each extra year of experience |
| b (intercept) | where the line crosses the vertical axis: the baseline salary at zero years of experience |
In the course reading the same model is written with weights: y = w1x1 + w0, where w1 plays the role of the slope and w0 the role of the intercept. For example, if w1 = 2 and w0 = 3, then for x1 = 5 the prediction is 25 + 3 = 13.
The algorithm uses the same learning loop introduced in Module 1:
The result is the line that best fits the overall pattern, minimising the average prediction error. Squaring the distances means points far from the line count much more than points close to it.
The lecture then compares two figures side by side:
With 10 or 20 features the model is a hyperplane. We cannot draw it any more, but the idea is the same: find the best flat surface through the data points, just in more dimensions. The general formula is:
y = w1x1 + w2x2 + ... + wp*xp + w0 + e
Here each wi is the weight of feature xi, w0 is the intercept (the prediction when every feature equals 0) and e is the noise the model cannot explain.
In scikit-learn both versions are handled by the same class, LinearRegression. The only difference is how many feature columns you pass in when you call fit.
A small, made-up salary table (salary in millions per month) shows the whole workflow. Note the double brackets: fit expects a table of features, even if there is only one column.
import pandas as pd
from sklearn.linear_model import LinearRegression
data = pd.DataFrame({
"Experience": [1, 2, 3, 4, 5],
"Education": [12, 16, 12, 16, 18],
"Salary": [12, 14, 16, 18, 20],
})
# Simple linear regression: one feature
simple = LinearRegression()
simple.fit(data[["Experience"]], data["Salary"])
print(simple.coef_, simple.intercept_)
new = pd.DataFrame({"Experience": [6]})
print(simple.predict(new))
# Multiple linear regression: same class, two feature columns
multi = LinearRegression()
multi.fit(data[["Experience", "Education"]], data["Salary"])
print(multi.coef_)
Expected output: the simple model has a slope of about 2 and an intercept of about 10, because every extra year adds exactly 2 to the salary in this toy data. The prediction for 6 years is about 22. The multiple model prints two coefficients, one per feature column; here the Education weight is close to 0 because Experience alone already explains the salaries perfectly.
data["Experience"] is a Series; scikit-learn expects a 2D table, so write data[["Experience"]].LinearRegression; only the number of feature columns changes.LinearRegression handles both simple and multiple regression.Flashcards · 2.1 Linear Regression
9 wordsAn algorithm that predicts a continuous number by fitting the straight line or flat surface that best matches the data.
How much the predicted target changes when the feature increases by one unit.
The predicted value when every feature equals zero; where the line crosses the vertical axis.
The line whose average squared distance to the data points is the smallest possible.
The average of the squared differences between predictions and actual values, used as the loss to minimise.
Linear Regression with exactly one input feature; the model is a straight line in 2D.
Linear Regression with two or more input features used together; the model is a plane or hyperplane.
The flat surface a linear model forms when there are many features; it cannot be drawn but works the same way.
The scikit-learn class for both simple and multiple Linear Regression; learned weights are in coef_ and intercept_.