Data Science, Editorial, Machine LearningThis article introduces support vector machines (SVM), and their roles in supervised machine learning. Author(s): Sujan Schirol, Roberto IriondoSVM is short for support vector machine. It can be used to solve classification and regression problems, but it’s primarily used for machine learning (ML) classification problems. SVM models are a type of supervised machine-learning technique that allows us to classify data points using previously classified data. The companion resources to this article can be found either on Google Colab or Github.What is Classification in Machine Learning?Classification is a supervised ML task that requires machine learning algorithms that learn how to assign a class label to examples from a problem domain. It is easy to understand how email classification works. There are many types of machine learning tasks and special approaches that can be applied for every [1].. There are four types of classification tasks that we may encounter in machine learning. Each type of task will need input data, labels and specialized approaches to modeling. There are four types of classification tasks:Binary Classification: Two class labels, spam/not-spam, 0/1.Multiclass Classification: More than two class labels face classification.Multi-Label Classification: Each instance can be assigned with multiple categories.Imbalanced Classification: The number of instances in each class is unequal.If you have not checked out yet, we recommend our K-nearest neighbor tutorial, as it is one of the basic machine learning algorithms to learn classification in supervised machine learning.A support vector machine is in charge of building a hyperplane or set of hyperplanes in a high or infinite-dimensional space, which, in turn, can be used for classification, regression, or other tasks like outliers detection.Intuitively, a good separation is achieved by the hyperplane with the most significant distance to the nearest training-data point of any class (so-called functional margin) since, in general, the larger the margin, the lower the generalization error of the classifier [2].Terminology Alert1. Support vector2. Hyperplane3. Functional Margin, also known as margin (from now on)4. Linearly Separatable5. The terminologies that are used to describe non-linearly separable machine learning algorithms can make them sound scary and complicated. However, they are straightforward to understand and derived from day-to-day life events.Let’s understand each of the terminologies of SVM with classification examples.Functionality of SVMThe image represents how an ideal SVM classifier looks when we visualize it on a 2-d plane. The colors red and blue mark the two types of SVM classifiers. It will become clearer why they are marked as positive or negative in the future. SVM generates a plane with 2-dimensional features, and a hyperplane with multidimensional features. This is called a decision border. It helps to classify the input data points based on their values. You can generate n number of hyperplanes, each one able to separate classes and determine the best. Margin and support vectors are used to determine the optimal hyperplane. The support vectors are the nearest points that the marginal planes pass through. This is called the margin. The SVM algorithm works by maximising the margin. There are two kinds of data distribution classification problems: linearly and non-linearly. If there is at least one line on the plane that has all of the blue and red points [3].Figure 2, then two sets can be considered linearly separate. Non-linearly separated data points are those which cannot be separated by a straight line, or with a straight hyperplane. SVM kernels can convert low dimensions into high dimensions to make data points linearly separate in such cases. For further information, check out the video shown below.Source: Video by udiprodSVM Use CaseFace DetectionText and Hypertext CategorizationClassification of ImagesBioinformaticsRemote Homology DetectionHandwriting DetectionGeneralized Predictive ControlMath behind SVMThe equation of a hyperplane is given by:Figure 3: Equation of a hyperplane.where the output y indicates whether it is in a positive class or the negative class. The matrix w that represents the parameters of the hyperplane and the coefficient of the x input data. The intercept of the hyperplane is b. The output of the classifier equation would not be positive if any of its points were located exactly at the decision border. Figure 6: Equation of decision boundary. We shall go ahead and derive why the output of the equation is either positive or negative.Figure 8: Derivation of the equation representation.Consider the above problem where the decision boundary passes through the origin and hence intercept is zero and its slope is +1. Each class is represented by a single hyperplane data point. Substituting values in equation of hyperplane. Figure 9: Value substitution within the equation. Next, maximize the margin to determine the best decision boundary. The negative support vector is point x1, and the positive vector is point x2. Simply, the margin is simply the difference between points x1 to x2. This equation can be solved by solving linear equations. Two equations have two unknowns mathematically (x1 andx2). The equivalent would be to divide both sides by the length of w, that is, the magnitude of the norm of w. Figure 11: Magnitude is 1 equivalent to the scalar 1. The equation for margin is now known as 12: Magnitude. It is an optimization function which must be maximized by using algorithms such gradient descent. Optimizing algorithms that find the minimum local to a problem are the most effective. To make it easier, minimize the complexity.
Introduction to Support Vector Machines (SVM), Machine Learning
