You are viewing a public preview
Sign in to save your progress, do assignments, join discussions and get feedback.
Between two classes of data you can draw infinitely many separating lines. SVM picks the line with the widest margin — that is, the one as far as possible from the nearest points of both classes.
Why is a wide margin good? Because it creates a safety buffer. A new data point that shifts slightly still lands on the correct side; with a boundary hugging the edge, even small noise is enough to cause a misclassification.
Only the points closest to the boundary determine the position of the hyperplane — these are the support vectors. Delete a point far away and the boundary does not change; delete a support vector and the boundary changes immediately.
A practical consequence: SVM depends on only a small fraction of the data, so it is memory-efficient at prediction time.
When the data cannot be separated by a straight line, SVM maps it into a higher-dimensional space where a separating hyperplane exists. The subtle point: the algorithm never actually computes the coordinates in the new space — it only computes inner products through the kernel function, so the cost stays manageable.
| Kernel | Use when |
|---|---|
linear | Data is linearly separable, many features |
rbf (default) | Curved boundary, shape unknown in advance |
poly | Polynomial relationships |
from sklearn.svm import SVC
model = SVC(kernel="rbf", C=1.0, gamma="scale")
model.fit(x_train, y_train)
The C parameter controls the tradeoff: large C = heavy penalty for misclassification, narrow margin, prone to overfitting. Small C = tolerate a few errors in exchange for a wide margin and better generalization.
Forgetting to standardize the data before using SVM. SVM is based on distances, so a feature with a large scale completely dominates all the others. Of all the models covered so far, this is the one most sensitive to feature scale.
By the end of this lesson you can:
linear, rbf, poly) for a situationKey idea: SVM picks the separating boundary with the widest safety margin, which depends only on the support vectors; kernels extend it to curved boundaries, and it only works well on standardised features.
Between two classes of data you can draw infinitely many separating lines. SVM chooses the line with the widest margin: the one as far as possible from the nearest points of both classes.
A wide margin creates a safety buffer. A new data point that shifts slightly still lands on the correct side. With a boundary hugging one class, even small noise can cause a misclassification. That is why SVM prefers the maximum-margin hyperplane rather than just any line that separates the training data.
Only the points closest to the boundary determine where the hyperplane is: these are the support vectors.
Practical consequence: SVM depends on only a small fraction of the data, so it is memory efficient when making predictions.
When the data cannot be separated by a straight line, SVM maps it into a higher-dimensional space where a separating hyperplane exists. The subtle point: the algorithm never actually computes the coordinates in the new space. It only computes inner products (a measure of similarity between pairs of points) through the kernel function, so the computational cost stays manageable.
| Kernel | Use when |
|---|---|
linear | data is linearly separable, many features |
rbf (default) | curved boundary, shape unknown in advance |
poly | polynomial relationships |
from sklearn.svm import SVC
model = SVC(kernel="rbf", C=1.0, gamma="scale")
model.fit(x_train, y_train)
C controls the trade-off between a wide margin and training errors:
| C | Margin | Risk |
|---|---|---|
| large | narrow; heavy penalty for every misclassified training point | overfitting |
| small | wide; tolerates a few errors | usually better generalisation |
C is a hyperparameter: you set it before training, and it is usually tuned with cross-validation (Module 3).
SVM is based on distances, so a feature with a large scale completely dominates the others. Of all the models covered so far, SVM is the most sensitive to feature scale. Standardising the data (for example with StandardScaler) is the most important preprocessing step before using SVM.
A dataset has two features: income (in VND, values in the tens of millions) and num_loans (0 to 5). Compare SVM with and without standardisation.
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
raw_svm = SVC(kernel="rbf", C=1.0)
scaled_svm = make_pipeline(StandardScaler(), SVC(kernel="rbf", C=1.0))
raw_svm.fit(x_train, y_train)
scaled_svm.fit(x_train, y_train)
print(raw_svm.score(x_test, y_test), scaled_svm.score(x_test, y_test))
Reasoning step by step:
income; a difference of 3 loans is invisible next to a difference of 10,000,000 VND.num_loans and its score usually suffers.StandardScaler inside a pipeline, both features have mean 0 and standard deviation 1, so both influence the margin.num_loans carries useful information. Putting the scaler inside the pipeline also means it is fitted on the training data only.linear, rbf (default) and poly kernels fit different boundary shapes.Flashcards · Support Vector Machines
8 wordsSVM's rule for choosing a boundary: the one furthest from the nearest points of both classes.
The training points closest to the boundary; removing one changes the hyperplane, removing others does not.
A function computing similarity (inner products) as if data were in a higher-dimensional space, without building that space.
An RBF kernel setting that controls how far the influence of a single training point reaches.
Penalty for misclassified training points: large C narrows the margin and risks overfitting; small C widens it.
Rescaling features to mean 0 and standard deviation 1 so no feature dominates SVM's distances.
A kernel for boundaries that follow polynomial relationships between features.
SVM needs only its support vectors at prediction time, a small fraction of the training data.