Đại học Kinh tế Quốc dân – National Economics UniversityMICRO4ASIA
🏆 Arena
Sign inSign up
MICRO4ASIAThe six micro-credentials

Courses are delivered on the Limio learning platform. Limio

← Fundamental of Machine Learning

You are viewing a public preview

Sign in to save your progress, do assignments, join discussions and get feedback.

Sign in
Module 2: Core Machine Learning Algorithms

2.2 Logistic Regression

Print / Save as PDF

2.2 Logistic Regression

After this unit you will: read a sigmoid graph and understand why Logistic Regression is a classification algorithm.

Key points

  • Logistic Regression is a classification problem, even though its name contains the word “regression”.
  • It uses the sigmoid function 1 / (1 + e^−(ax+b)) to produce a probability in the range (0, 1).
  • The probability is then compared with a threshold (usually 0.5): above the threshold the label is 1 (Senior), below it the label is 0 (Junior).
  • It is the threshold comparison step that produces the label 0 or 1, not the sigmoid.

Study notes

By the end of this lesson you can:

  • Explain why Logistic Regression is a classification algorithm despite its name
  • Describe the sigmoid function and why its output can be read as a probability
  • Turn a predicted probability into a label (Junior = 0, Senior = 1) using a threshold
  • Read the shape of the sigmoid curve on the salary and job level chart
  • Name the scikit-learn class that implements the algorithm

Key idea: Logistic Regression squeezes any input into a probability between 0 and 1 with the S-shaped sigmoid curve, and then a threshold (usually 0.5) turns that probability into a class label.

A classification algorithm with a confusing name

The lecture is very clear about a point that trips up many beginners: despite the word "regression" in its name, Logistic Regression is a classification algorithm. It predicts categories, not continuous numbers.

The running example uses salary on the horizontal axis and job level on the vertical axis, where 0 means Junior and 1 means Senior. The data points cluster near the bottom of the chart at low salaries (juniors) and near the top at high salaries (seniors). The question is: given a salary, is this person Junior or Senior?

Why a curve instead of a straight line

A straight line, as in Linear Regression, can shoot off towards plus or minus infinity. That makes no sense if we want an output that represents a probability, which must stay between 0 and 1. So Logistic Regression uses an S-shaped curve called the sigmoid function:

sigmoid(z) = 1 / (1 + e^(-z)), where z = a*x + b

Whatever value of x we plug in, the output is always strictly between 0 and 1. Two useful facts:

  • When z = 0, e^0 = 1, so the output is 1 / (1 + 1) = 0.5.
  • Large positive z gives an output close to 1; large negative z gives an output close to 0.

We read the output as a probability. If the sigmoid outputs 0.9 for a particular salary, the model is saying there is a 90% probability that this person is Senior.

From probability to label: the threshold

The sigmoid alone does not produce 0 or 1. The final label comes from comparing the probability with a threshold, usually 0.5:

Predicted probability of SeniorLabel with threshold 0.5
above 0.51 (Senior)
below 0.50 (Junior)

This comparison step is what turns a probability into a category. The threshold is a choice: if you raise it (for example to 0.9), the model needs to be much more confident before it says "Senior", so some people previously labelled Senior become Junior, even though the model itself has not changed.

Reading the shape of the curve

The curve is very flat and close to 0 for low salaries, rises sharply through the middle, and flattens again near 1 for high salaries. This matches our intuition:

  • very low salaries are strongly associated with being Junior,
  • very high salaries are strongly associated with being Senior,
  • in the middle there is a transition zone where the model is less certain.

In scikit-learn the algorithm is implemented as LogisticRegression, and again, it is for classification problems.

Worked example

Suppose a trained model has learned z = 0.2*salary - 6, with salary in millions per month (made-up numbers for practice). Compute the probability of Senior for three employees and apply the 0.5 threshold.

import math

def sigmoid(z):
    return 1 / (1 + math.exp(-z))

for salary in [10, 30, 50]:
    z = 0.2 * salary - 6
    p = sigmoid(z)
    label = 1 if p > 0.5 else 0
    print(salary, round(p, 3), label)

Expected output:

10 0.018 0
30 0.5 0
50 0.982 1

Step by step: for salary 10, z = -4, so the probability is tiny and the label is Junior. For salary 30, z = 0, so the probability is exactly 0.5; it is not above the threshold, so the label is 0 (the transition point). For salary 50, z = 4, the probability is about 0.98 and the label is Senior.

In scikit-learn the same steps are done for you: model.predict_proba(x) returns the probabilities and model.predict(x) returns the labels using the 0.5 threshold.

from sklearn.linear_model import LogisticRegression

model = LogisticRegression()
model.fit(x_train, y_train)          # y_train holds 0 (Junior) or 1 (Senior)
print(model.predict_proba(x_test)[:3])
print(model.predict(x_test)[:3])

Common mistakes

  • Treating it as a regression model because of the name. Logistic Regression answers "which category?", not "how much?".
  • Thinking the sigmoid outputs the label. The sigmoid outputs a probability; the threshold comparison produces the 0/1 label.
  • Assuming the threshold is fixed at 0.5. It is the usual default, but changing it changes the labels without retraining the model.
  • Expecting outputs of exactly 0 or 1 from the sigmoid. The curve gets very close to 0 and 1 but stays strictly inside the range (0, 1).

Summary

  • Logistic Regression is a classification algorithm (Junior vs Senior from salary).
  • The sigmoid 1 / (1 + e^(-(a*x + b))) maps any input to a probability in (0, 1); at z = 0 it gives 0.5.
  • A threshold, usually 0.5, turns the probability into a label 0 or 1.
  • The S-shaped curve shows confident predictions at both ends and an uncertain zone in the middle.
  • In scikit-learn, use LogisticRegression; predict_proba gives probabilities and predict gives labels.

Flashcards · 2.2 Logistic Regression

8 words
WordMeaningExample
  • Logistic Regression

    A classification algorithm that predicts the probability of a class with the sigmoid function, then assigns a label using a threshold.

    Predicting Junior (0) or Senior (1) from salary.

  • Sigmoid function

    The S-shaped function 1 / (1 + e^(-z)) that maps any number to a value strictly between 0 and 1.

    sigmoid(0) = 0.5; sigmoid(4) is about 0.98.

  • Threshold

    The cut-off compared with the predicted probability to decide the label; usually 0.5.

    Probability 0.9 > 0.5, so the label is Senior.

  • Predicted probability

    The sigmoid output, read as how likely the sample belongs to the positive class.

    0.9 means a 90% probability that the person is Senior.

  • Binary classification

    A task with exactly two possible categories, often coded as 0 and 1.

    Junior = 0, Senior = 1.

  • Transition zone

    The middle part of the sigmoid curve where the probability rises steeply and predictions are least certain.

    Mid-range salaries with probabilities around 0.5.

  • LogisticRegression

    The scikit-learn class for Logistic Regression; used for classification problems despite its name.

    LogisticRegression().fit(x_train, y_train)

  • predict_proba()

    A classifier method that returns the predicted probability of each class for every sample.

    model.predict_proba(x_test)[:, 1] gives P(Senior).