Enhancement of Discriminative Embedded Clustering for Clustering High Dimensional Data using Hub Concept

Abstract

We often face very high dimensional data in many real applications. Many dimensions are not always helpful or may even affect the performance of the subsequent clustering algorithms. For dealing with this problem one way is to first reduce dimensionality and then apply clustering. But if we consider the requirement of dimensionality reduction during the process of clustering and vice versa then the performance of clustering can be improved. Discriminative Embedded Clustering (DEC) combines clustering and subspace learning. It has two main objective functions, first is dimensionality reduction and second is clustering.

In high dimensional data some data points are included in many more k-nearest-neighbor lists compared to other points. These points are called hubs. The tendency of high dimensional data to contain hubs is called hubness. Hubs are closer to all the other points as they are situated near cluster centeres. It is proved that major hubs can be effectively used as cluster prototypes. Use of hubness for clustering leads to enhancement over centroid-based approaches. Therefore, the aim of this paper is to design a system for clustering high dimensional data by using Discriminative Embedding Method and Hub based clustering.