Author(s: LA Tran Deep Learning We need to raise the potentials of those that have not been paid attention. Clustering refers to grouping multiple-dimensional data into related groups. K-means Clustering (SOM) and Self-Organizing Maps (SOM) are two classic examples of clustering algorithms. These algorithm explanations can be found in many places. Let me now introduce you to a more efficient algorithm for clustering, but which seems few researchers are paying attention to: Centroid Neural Network For Unsupervised Competitive Learning. Click here for a more detailed look at the paper. Centroid Neural Network Results — Source: Author. Let’s now get to work! Centroid Neural Network – (CentNN). To avoid confusing with Convolution Neural Network I will use “CentNN”. CentNN, an unsupervised competition learning algorithm, is based on the classic k-means algorithm for clustering. It estimates the centroids from the cluster groups that are related in the training date. CentNN does not require a set number of clustering iterations or a learning curve. The paper presents simulation results for image compression and clustering problems. These results show that CentNN is faster than other algorithms and has compatible clustering quality. Other algorithms can give inconsistent results depending upon the initial learning coefficient value and how many iterations are used. The Centroid Neural-Network Algorithm. Figure from paper. The CentNN algorithm is shown in the above image. If you get bored when looking at all those mathematical equations, don’t worry, I am going to make it more understandable with a secondary-school-maths example. Weight Update This is the core algorithm. It involves how to efficiently calculate the centroid for a set of data points, when one data point changes in or out. It’s easy to find the average value of all numbers if we have ones. What happens if there is another integer? Is it necessary to add all 101 these numbers again? No! The answer is NO! The process is similar when one integer is removed to create 99 numbers. These equations are explained in the paper. Weight Update Equations Visual Explanation Example Given the 30 dataset shown below, we want to group them into three clusters. Photo by Author Epoch 0. Initialization. Find one centroid for every data point. Then, divide c into two weights, w1, and w2, with small E. E = 0. 05 (Image by Author) Before we move on, let us talk about “loser and winner neurons”. Enter a data value x. The winner neuron will be the one closest to the x. Enter a data value x for epoch 1. If x is the nearest weight to x (n-1), then the winner neuron is the weight (w) that is closest to x. However, in epoch 2 x3 is, the loser neuron is the weight (w). Update w1 30 wins, and update w1. Image by Author. After epoch 0, there are always two centroids. Also, information about clustered points. Keep it simple: x1 is, w1 win x2 comes. w2 winsx3 comes. w1 won Epoch 1. Find the winner neuron of each x in this X. Image by Author. X1 arrives, w2 loses, w1 wins, loser = 1, update weights. W2 wins, x2 arrives, x3 wins, w1 loses, loser += 1, update weights. After Epoch 1, there is 1 data, and the loss neurons increase by 1. We can see that x1 is still closer to W2 than to W1, however, at Epoch 0, the cluster of x1 had been updated to match w1 and now it has 1. You can see that x1 is here, and w2 wins the x2 come, x3 wins the x3 come, w1 win… x comes. Epoch 2: Continue to find the winning neuron for every x in the X. Image by the Author. x1 arrives, w2 win x2 is, w2 won x3 comes and w1 wins Epoch 2. Keep looking for the winner neuron for each x in X. Let’s see if we can get the desired amount of clusters. It is not yet. Next, we split the centroid with the least error using a small E. Image by Author Repeat the procedure starting at epoch 1. Epoch 3: Images By Author Epoch 4 and Epoch 5 are the same. We also achieve the required number of clusters. It’s time for the party! GIF via Tenor Below are the differences between CentNN and K-Means clustering in image compression. CentNN is a better choice than K-Means clustering when it comes to PSNR. With PSNR , we can clearly see the difference. 53 is better than PSNR 46.. 35, respectively. 35 respectively. Image compression results — Image by CentNN Author CentNN Implementation. You can see my Python implementation of CentNN here. Please give this a star if you find it helpful. You are also welcome to check out my Facebook page, which you can use for machine learning: Diving into Machine Learning. This post introduces Centroid Neural Network, an efficient clustering algorithm. CentNN updates weights whenever new data arrives, and K-means Clustering updates the centroids after each iteration. This is CentNN’s main feature. CentNN sets the number to 2 and then increases it one at a time until the desired number. Although the CentNN algorithm doesn’t guarantee convergence to the global minimal solution as other unsupervised algorithms, it does guarantee convergence to a local minim. Centroid Neural Network – An Efficient, Stable Clustering Algorithm originally appeared in on Medium. People are continuing to the conversation by responding and highlighting this story. Published via
Centroid Neural network: A stable and efficient clustering algorithm
