Table of Contents

SKMeans

Learn features from data

KMeans

SKMeans is an object that facilitates learning features from data. It is very similar to KMeans but with 2 distinctions.

  1. Instead of taking the Euclidian distance to each center of each cluster like KMeans does, SKMeans takes the angular distance of the normalised vector in high dimension. This is also named cosine similarity.

  2. Once the clusters are found, the distances are encoded with an ‘alpha’ function, in effect promoting sparser representations where smaller similarities are penalised.

In effect, SKMeans is mostly used to learn features in a higher dimension space than the original data, with the assumption that it would help untangle near clusters.

Spherical KMeans @Machine Learning Catalogue

A terse definition of the algorithm.

https://machinelearningcatalogue.com/algorithm/alg_spherical-k-means.html

Classic vs Spherical KMeans

A quite thorough explanation of the difference between "classic" and spherical KMeans.

https://stats.stackexchange.com/questions/63558/difference-between-standard-and-spherical-k-means-algorithms

Coates and Ng - Learning Feature Representations with K-means

The original paper describing the implementation of encoded activations in feature learning.

https://www-cs.stanford.edu/~acoates/papers/coatesng_nntot2012.pdf