Support Vector Machines- An easy interpretation of categorizing inseparable data

In Machine learning the Support Vector Models are used for supervised learning based on associated learning algorithms for classification and Regression based analysis.

If you have examples of data marked as belonging to one of the classes, SVM training algorithm assigns new data one of the classes as non-probabilistic binary linear classifier.

An SVM model is a representation of the data points in space, mapped to each class so that the examples of the separate classes are divided by a clear gap that is as wide as possible. Examples of new data are then mapped into that same space and predicted to belong to a class based on the side of the gap on which they fall.

In addition to linear classification, SVM can also categorize non-linear data using a kernel –trick.

When supervised learning is not possible and the data is unlabeled, data clustering approach is taken to categorize the data into one of the category, this learning is unsupervised and applies support vector clustering algorithm.

This support vector algorithm is used to categorize the unlabeled data into its nearest category.

When the original problem is stated in finite dimension plane, the problem to categorize the data may not be possible in linear space.

It was thus proposed to have a higher dimension in which data can be separated and classified, to keep the computational load low so that that the pair of input vectors may be computed easily in space by defining in terms of Kernel Function.

Kernel Trick

Kernel trick is actually giving a solution in a plane which doesn’t exist originally in problem.

If we get a set of data as below we see that these cannot be categorized linearly with a straight line.

If we have the values as mentioned in image below of x-y axis, and plot them on 2D plane of x-y, we see that it cannot be categorized.

To add a new plane to it, we need to have a value for 3rd dimension which will easily separate out the red and green data point’s category.

We will try 3 equations here.

1) x + y

2) xy

3) x**2

As mentioned in the table above we can clearly see that xy formula has separated out red and green data points with values 0 and 2 respectively.

Now add 3rd dimension as z axis and plot the values of xy on it. We can see that these are easily separable with red data points going down and green going up.

To show this in linear space we know the solution is between xy=0 and xy=2 i.e.

xy=1 , y=1/x, which is exactly the shape of line shown in image above.

It is to be noted that working in a higher-dimensional feature space increases the generalization error of support-vector machines, although given enough samples the algorithm still performs well.

Thanks for reading!