You are viewing a public preview
Sign in to save your progress, do assignments, join discussions and get feedback.
After this unit you will: read a sigmoid graph and understand why Logistic Regression is a classification algorithm.
By the end of this lesson you can:
Key idea: Logistic Regression squeezes any input into a probability between 0 and 1 with the S-shaped sigmoid curve, and then a threshold (usually 0.5) turns that probability into a class label.
The lecture is very clear about a point that trips up many beginners: despite the word "regression" in its name, Logistic Regression is a classification algorithm. It predicts categories, not continuous numbers.
The running example uses salary on the horizontal axis and job level on the vertical axis, where 0 means Junior and 1 means Senior. The data points cluster near the bottom of the chart at low salaries (juniors) and near the top at high salaries (seniors). The question is: given a salary, is this person Junior or Senior?
A straight line, as in Linear Regression, can shoot off towards plus or minus infinity. That makes no sense if we want an output that represents a probability, which must stay between 0 and 1. So Logistic Regression uses an S-shaped curve called the sigmoid function:
sigmoid(z) = 1 / (1 + e^(-z)), where z = a*x + b
Whatever value of x we plug in, the output is always strictly between 0 and 1. Two useful facts:
We read the output as a probability. If the sigmoid outputs 0.9 for a particular salary, the model is saying there is a 90% probability that this person is Senior.
The sigmoid alone does not produce 0 or 1. The final label comes from comparing the probability with a threshold, usually 0.5:
| Predicted probability of Senior | Label with threshold 0.5 |
|---|---|
| above 0.5 | 1 (Senior) |
| below 0.5 | 0 (Junior) |
This comparison step is what turns a probability into a category. The threshold is a choice: if you raise it (for example to 0.9), the model needs to be much more confident before it says "Senior", so some people previously labelled Senior become Junior, even though the model itself has not changed.
The curve is very flat and close to 0 for low salaries, rises sharply through the middle, and flattens again near 1 for high salaries. This matches our intuition:
In scikit-learn the algorithm is implemented as LogisticRegression, and again, it is for classification problems.
Suppose a trained model has learned z = 0.2*salary - 6, with salary in millions per month (made-up numbers for practice). Compute the probability of Senior for three employees and apply the 0.5 threshold.
import math
def sigmoid(z):
return 1 / (1 + math.exp(-z))
for salary in [10, 30, 50]:
z = 0.2 * salary - 6
p = sigmoid(z)
label = 1 if p > 0.5 else 0
print(salary, round(p, 3), label)
Expected output:
10 0.018 0
30 0.5 0
50 0.982 1
Step by step: for salary 10, z = -4, so the probability is tiny and the label is Junior. For salary 30, z = 0, so the probability is exactly 0.5; it is not above the threshold, so the label is 0 (the transition point). For salary 50, z = 4, the probability is about 0.98 and the label is Senior.
In scikit-learn the same steps are done for you: model.predict_proba(x) returns the probabilities and model.predict(x) returns the labels using the 0.5 threshold.
from sklearn.linear_model import LogisticRegression
model = LogisticRegression()
model.fit(x_train, y_train) # y_train holds 0 (Junior) or 1 (Senior)
print(model.predict_proba(x_test)[:3])
print(model.predict(x_test)[:3])
LogisticRegression; predict_proba gives probabilities and predict gives labels.Flashcards · 2.2 Logistic Regression
8 wordsA classification algorithm that predicts the probability of a class with the sigmoid function, then assigns a label using a threshold.
The S-shaped function 1 / (1 + e^(-z)) that maps any number to a value strictly between 0 and 1.
The cut-off compared with the predicted probability to decide the label; usually 0.5.
The sigmoid output, read as how likely the sample belongs to the positive class.
A task with exactly two possible categories, often coded as 0 and 1.
The middle part of the sigmoid curve where the probability rises steeply and predictions are least certain.
The scikit-learn class for Logistic Regression; used for classification problems despite its name.
A classifier method that returns the predicted probability of each class for every sample.