🔍 Support Vector Machine (SVM)

SVM is a supervised machine learning algorithm used for classification problems. It aims to find a hyperplane that best separates two classes.

💡 Difference:

• Logistic Regression: Probabilistic approach
• SVM: Statistical approach

🧩 SVM Hyperparameters: C and gamma

🔧 C – Regularization Parameter

• Controls the trade-off between a smooth decision boundary and classifying training points correctly.
• Low C:
  • More regularization (allows wider margin).
  • Might misclassify some training points to keep the margin large.
• High C:
  • Less regularization (narrow margin).
  • Tries to classify all training examples correctly, potentially overfitting.

🔧 gamma – Kernel Coefficient

• Relevant for RBF, polynomial, and sigmoid kernels.
• Defines how far the influence of a single training example reaches.
• Low gamma:
  • Broad influence → more generalized model.
• High gamma:
  • Narrow influence → may overfit to training data.

🔢 Sample Code: Using C and gamma in RBF Kernel

from sklearn.svm import SVC
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# Load dataset
data = load_iris()
X = data.data
y = data.target

# Binary classification for SVM example
X = X[y != 2]
y = y[y != 2]

# Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Train SVM with custom C and gamma
model = SVC(kernel='rbf', C=10, gamma=0.1)
model.fit(X_train, y_train)

# Predict and evaluate
y_pred = model.predict(X_test)
print("Accuracy:", accuracy_score(y_test, y_pred))

📏 Types of SVM

1. Linear SVM

• Used when data is perfectly linearly separable.