2.8 SVM: The Kernel Trick
[video] /api/lesson-media/videos/9b417aa2-5e9f-4ae3-a9db-2a3b7f03f372-1794614893962-4cd610080c52b8e8.mp4
[markdown] content available only on the platform
[markdown] content available only on the platform
Flashcards · 2.8 SVM: The Kernel Trick
Non-linearly separable data — Data whose classes cannot be separated by any straight line or flat hyperplane in the original space.
Green circles in the centre, stars in a ring around them.
Kernel trick — Mapping data into a higher-dimensional space where a flat hyperplane can separate classes that were not separable before.
Lifting 2D points into a 3D box, split by a plane.
Linear kernel — The ordinary straight-line SVM; suitable when the data is close to linearly separable.
SVC(kernel='linear')
Polynomial kernel — A kernel that produces curved boundaries of a chosen polynomial degree.
SVC(kernel='poly', degree=3)
RBF kernel — Radial basis function (Gaussian) kernel; very flexible and the most common default choice for SVM.
SVC(kernel='rbf')
Feature mapping — Creating new features from old ones so the classes become separable, e.g. adding x squared.
x -> (x, x^2) separates -2, 0, 2 from -5, 5.
Feature scaling for SVM — Standardising features before SVM so no large-scale feature dominates the distance calculations.
make_pipeline(StandardScaler(), SVC())
Distance-based algorithm — An algorithm whose decisions depend on distances between points, making it sensitive to feature scale.
SVM, because its margin is a distance.
