Improving Classification Performance of Fully Connected Layers by Fuzzy Clustering in Transformed Feature Space

Abstract

Fully connected (FC) layers are used in almost all neural network architectures ranging from multilayer perceptrons to deep neural networks. FC layers allow any kind of symmetric/asymmetric interaction between features without making any assumption about the structure of the data. However, success of convolutional and recursive layers and findings of many studies have proven that the intrinsic structure of a dataset holds a great potential to improve the success of a classification problem. Leveraging clustering to explore and exploit this intrinsic structure in classification problems has been the subject of various studies. In this paper, we propose a new training pipeline for fully connected layers which enables them to make more accurate classification predictions. The proposed method aims to reflect the clustering patterns in the original feature space of the training dataset to the transformed feature space created by the FC layer. In this way, we intend to enhance the representation ability of the extracted features and accordingly increase the classification accuracy. The Fuzzy C-Means algorithm is employed in this study as the clustering tool. To evaluate the performance of the proposed method, 11 experiments were conducted on 9 benchmark UCI datasets. Empirical results show that the proposed method works well in practice and gives higher classification accuracies compared to a regular FC layer in most datasets.