Output [0.26860518]
Regression with SVM As discussed earlier, SVM is used for both classification and regression problems. Scikit-
learn’s method of Support Vector Classification (SVC) can be extended to solve regression
problems as well. That extended method is called Support Vector Regression (SVR).
Basic similarity between SVM and SVR The model created by SVC depends only on a subset of training data. Why? Because the
cost function for building the model doesn’t care about training data points that lie outside
the margin.
Whereas, the model produced by SVR (Support Vector Regression) also only depends on
a subset of the training data. Why? Because the cost function for building the model
ignores any training data points close to the model prediction.
Scikit-learn provides three classes namely
SVR, NuSVR and LinearSVR as three
different implementations of SVR.
SVR It is Epsilon-support vector regression whose implementation is based on
libsvm . As
opposite to
SVC There are two free parameters in the model namely ‘
C ’ and ‘
epsilon ’.
epsilon: float, optional, default = 0.1
It represents the epsilon in the epsilon-SVR model, and specifies the epsilon-tube within
which no penalty is associated in the training loss function with points predicted within a
distance epsilon from the actual value.
Rest of the parameters and attributes are similar as we used in
SVC .
Implementation Example
Scikit-Learn
72
Following Python script uses
sklearn.svm.SVR class
:
from sklearn import svm
X = [[1, 1], [2, 2]]
y = [1, 2]
SVRReg = svm.SVR(
kernel=’linear’, gamma=’auto’
)
SVRReg.fit(X, y)
Output SVR(C=1.0, cache_size=200, coef0=0.0, degree=3, epsilon=0.1, gamma='auto',
kernel='linear', max_iter=-1, shrinking=True, tol=0.001, verbose=False)
Now, once fitted, we can get the weight vector with the help of following python script:
SVRReg.coef_
