Class Note 3: Support Vector Machines (SVM)
Introduction to Support Vector Machines
Support Vector Machines (SVM) is a powerful supervised learning
algorithm used for both classification and regression tasks. The goal
of SVM is to find the hyperplane that best separates the data into
different classes while maximizing the margin between the classes.
SVM can handle linear and non-linear data using kernel functions.
Major Points
1. Hyperplane: In a binary classification problem, the
hyperplane is the decision boundary that separates the two
classes. The optimal hyperplane is the one that maximizes the
margin between the classes.
2. Kernel Trick: SVM can handle non-linear data by mapping
the input features into a higher-dimensional space using
kernel functions like Polynomial, Radial Basis Function (RBF),
and Sigmoid.
3. Regularization: The C parameter in SVM controls the trade-
off between maximizing the margin and minimizing the
classification error. A smaller C value creates a larger margin
but allows more misclassifications.
4. Multi-Class Classification: SVM can be extended to multi-
class classification using techniques like One-vs-One or One-
vs-All.
Use Cases
1. Image Classification: SVM can be used for image
classification tasks, such as identifying objects in images.
2. Text Classification: SVM is commonly used in natural
language processing for tasks like sentiment analysis and
spam detection.
3. Bioinformatics: In bioinformatics, SVM can be used for
protein classification and gene expression analysis.
Optimization Techniques
1. Kernel Selection: Choosing the right kernel function is
crucial for SVM performance. RBF is a popular choice for non-
linear data.
2. Parameter Tuning: The C and gamma parameters should be
tuned using cross-validation to achieve the best performance.
3. Feature Scaling: SVM is sensitive to the scale of input
features, so it is important to standardize or normalize the
data.
4. Handling Imbalanced Data: Techniques like class weighting
or SMOTE can be used to handle imbalanced datasets in SVM.
