You are viewing a public preview
Sign in to save your progress, do assignments, join discussions and get feedback.
After this unit you will be able to: explain the maximum margin and the role of support vectors.
By the end of this lesson you can:
Key idea: Among all lines that separate the classes, SVM picks the one with the widest margin (empty corridor) between them; only the closest points, the support vectors, hold that boundary in place.
The lecture answers the question from the previous lesson. The picture now shows:
The dashed lines mark a corridor on either side of the boundary. The width of that corridor is the margin: a safety buffer of empty space between the two classes, with the boundary running down the middle.
The core idea of SVM: out of all the infinitely many separating lines, choose the one that makes the margin as wide as possible. That is why SVM is called a maximum margin classifier.
Think of driving a car. If you drive right along the edge of the lane, even a tiny wobble takes you across the line. If you drive down the centre, you have room on both sides to absorb small errors.
A boundary with a wide margin is as far as possible from both groups, so a new data point would have to be quite unusual before it lands on the wrong side. A boundary that hugs one group is fragile; a boundary with a generous margin is robust, and robustness tends to give better performance on new, unseen data.
The red-circled points sit exactly on the edge of the margin, touching the yellow dashed lines. They are the points closest to the boundary, the hardest cases in the dataset.
A surprising fact: the support vectors are the only points that determine where the boundary goes.
They "support", or hold up, the boundary, hence the name.
Practical consequences:
Real data is rarely this tidy: some points sit on the wrong side and no line separates them perfectly. SVM implementations handle this with the hyperparameter C, which controls the trade-off between a wide margin and tolerating misclassified training points.
| C | Behaviour | Typical effect |
|---|---|---|
| small | keep the margin wide, accept a few mistakes | usually generalises better |
| large | classify every training point correctly, even with a narrow margin | risk of overfitting, as with deep decision trees |
Tuning C is a big part of getting good results with SVM.
One-dimensional data: class A has x = 0 and x = 1; class B has x = 5 and x = 6. Where does a maximum margin SVM put the boundary?
The same result with scikit-learn:
import numpy as np
from sklearn.svm import SVC
X = np.array([[0], [1], [5], [6]])
y = np.array([0, 0, 1, 1]) # 0 = class A, 1 = class B
model = SVC(kernel="linear", C=1000) # large C: (almost) no errors allowed
model.fit(X, y)
print(model.support_vectors_.ravel())
print(-model.intercept_[0] / model.coef_[0, 0])
Expected output: the support vectors [1. 5.] and a boundary at about 3.0 (the point where the decision function equals zero).
Flashcards · 2.7 SVM: Maximum Margin and Support Vectors
7 wordsThe width of the empty corridor between the decision boundary and the nearest points of each class.
A classifier that chooses, among all separating boundaries, the one with the widest margin; this describes SVM.
The training points closest to the boundary, lying on the margin edge; they alone fix the boundary's position.
A boundary far from both classes, so new points must be unusual to land on the wrong side.
SVM setting that trades margin width against training errors: small C allows mistakes, large C narrows the margin.
A trained SVM only needs its support vectors to make predictions, not the whole training set.
The scikit-learn class for SVM classification.