Đại học Kinh tế Quốc dân – National Economics UniversityMICRO4ASIA
🏆 Arena
Sign inSign up
MICRO4ASIAThe six micro-credentials

Courses are delivered on the Limio learning platform. Limio

← Fundamental of Machine Learning

You are viewing a public preview

Sign in to save your progress, do assignments, join discussions and get feedback.

Sign in
Module 2: Core Machine Learning Algorithms

2.1 Linear Regression

Print / Save as PDF

2.1 Linear Regression

After this unit you will: explain what Linear Regression does, and distinguish simple regression from multiple regression.

Key points

  • Linear Regression fits a straight line through a cloud of points to predict a continuous numeric value — for example, salary as a function of years of experience.
  • Simple Linear Regression: one input feature, and the model is a straight line.
  • Multiple Linear Regression: several features (for example Experience and Education), and the model is a plane (hyperplane).

y = ω₁x₁ + ω₂x₂ + … + ωₚxₚ + ω₀ + ε

Study notes

By the end of this lesson you can:

  • Explain what Linear Regression predicts and why it suits a continuous target such as salary
  • Read the line equation y = m*x + b and say what the slope and the intercept mean
  • Describe how the algorithm finds the best fit line by repeatedly reducing the mean squared error
  • Distinguish simple Linear Regression (one feature, a line) from multiple Linear Regression (several features, a plane or hyperplane)
  • Train both versions with the same scikit-learn class, LinearRegression

Key idea: Linear Regression assumes a roughly straight-line relationship between the features and a numeric target, and it learns the line (or flat surface) that keeps the average squared prediction error as small as possible.

The salary and experience example

The lecture opens with a scatter plot: salary on the vertical axis, years of experience on the horizontal axis. The points follow an upward diagonal trend, and a green straight line runs through them. That line is what the model learns. One red point sits above the general trend; no straight line passes through every point, so the model aims for the line that fits the overall pattern best.

Linear Regression is a good first algorithm for Module 2 because the idea is simple: as experience goes up, salary tends to go up in a consistent, linear way. Because the target (salary) is a continuous number and not a category, this is a regression problem.

The equation of the line

The learned line is the classic equation y = m*x + b:

SymbolMeaning in the example
ypredicted salary
xyears of experience
m (slope)how much salary increases for each extra year of experience
b (intercept)where the line crosses the vertical axis: the baseline salary at zero years of experience

In the course reading the same model is written with weights: y = w1x1 + w0, where w1 plays the role of the slope and w0 the role of the intercept. For example, if w1 = 2 and w0 = 3, then for x1 = 5 the prediction is 25 + 3 = 13.

How the line is found

The algorithm uses the same learning loop introduced in Module 1:

  1. Start with an initial guess for the slope and intercept.
  2. Measure the loss, usually the mean squared error (MSE): the average of the squared vertical distances between the line and the actual data points.
  3. Adjust the slope and intercept so that the loss goes down.
  4. Repeat until the loss stops improving.

The result is the line that best fits the overall pattern, minimising the average prediction error. Squaring the distances means points far from the line count much more than points close to it.

From a line to a plane: multiple Linear Regression

The lecture then compares two figures side by side:

  • Simple Linear Regression uses one feature to predict the target (experience predicts salary). Geometrically the model is a straight line in two-dimensional space.
  • Multiple Linear Regression uses two or more features at the same time (education and experience together predict salary). With two features the model becomes a flat plane in three-dimensional space.

With 10 or 20 features the model is a hyperplane. We cannot draw it any more, but the idea is the same: find the best flat surface through the data points, just in more dimensions. The general formula is:

y = w1x1 + w2x2 + ... + wp*xp + w0 + e

Here each wi is the weight of feature xi, w0 is the intercept (the prediction when every feature equals 0) and e is the noise the model cannot explain.

In scikit-learn both versions are handled by the same class, LinearRegression. The only difference is how many feature columns you pass in when you call fit.

Worked example

A small, made-up salary table (salary in millions per month) shows the whole workflow. Note the double brackets: fit expects a table of features, even if there is only one column.

import pandas as pd
from sklearn.linear_model import LinearRegression

data = pd.DataFrame({
    "Experience": [1, 2, 3, 4, 5],
    "Education":  [12, 16, 12, 16, 18],
    "Salary":     [12, 14, 16, 18, 20],
})

# Simple linear regression: one feature
simple = LinearRegression()
simple.fit(data[["Experience"]], data["Salary"])
print(simple.coef_, simple.intercept_)

new = pd.DataFrame({"Experience": [6]})
print(simple.predict(new))

# Multiple linear regression: same class, two feature columns
multi = LinearRegression()
multi.fit(data[["Experience", "Education"]], data["Salary"])
print(multi.coef_)

Expected output: the simple model has a slope of about 2 and an intercept of about 10, because every extra year adds exactly 2 to the salary in this toy data. The prediction for 6 years is about 22. The multiple model prints two coefficients, one per feature column; here the Education weight is close to 0 because Experience alone already explains the salaries perfectly.

Common mistakes

  • Using Linear Regression for categories. It predicts a continuous number. For a yes/no or junior/senior label you need a classifier such as Logistic Regression (next lesson).
  • Passing a single column with single brackets. data["Experience"] is a Series; scikit-learn expects a 2D table, so write data[["Experience"]].
  • Thinking multiple regression needs a different class. Both simple and multiple regression use LinearRegression; only the number of feature columns changes.
  • Expecting the line to pass through every point. The best fit line minimises the average error; individual points (like the red one above the trend) can still be far away.

Summary

  • Linear Regression predicts a continuous target by fitting a straight line through the data.
  • The line y = m*x + b has a slope (change in salary per year of experience) and an intercept (baseline salary at zero experience).
  • The model learns by repeatedly adjusting the line to reduce the mean squared error.
  • One feature gives a line, two features give a plane, many features give a hyperplane.
  • In scikit-learn, LinearRegression handles both simple and multiple regression.

Flashcards · 2.1 Linear Regression

9 words
WordMeaningExample
  • Linear Regression

    An algorithm that predicts a continuous number by fitting the straight line or flat surface that best matches the data.

    Predicting salary from years of experience.

  • Slope (m)

    How much the predicted target changes when the feature increases by one unit.

    m = 2: each extra year of experience adds 2 to predicted salary.

  • Intercept (b)

    The predicted value when every feature equals zero; where the line crosses the vertical axis.

    Baseline salary for someone with 0 years of experience.

  • Best fit line

    The line whose average squared distance to the data points is the smallest possible.

    The green line through the salary and experience scatter plot.

  • Mean squared error (MSE)

    The average of the squared differences between predictions and actual values, used as the loss to minimise.

    Errors 1 and 3 give MSE = (1 + 9) / 2 = 5.

  • Simple Linear Regression

    Linear Regression with exactly one input feature; the model is a straight line in 2D.

    Salary predicted from Experience only.

  • Multiple Linear Regression

    Linear Regression with two or more input features used together; the model is a plane or hyperplane.

    Salary predicted from Experience and Education.

  • Hyperplane

    The flat surface a linear model forms when there are many features; it cannot be drawn but works the same way.

    A model with 20 features fits a hyperplane in 21 dimensions.

  • LinearRegression

    The scikit-learn class for both simple and multiple Linear Regression; learned weights are in coef_ and intercept_.

    LinearRegression().fit(data[['Experience']], data['Salary'])