Stable feature selection using MRMR algorithm

Abstract

Feature selection methods help machine learning algorithms produce faster and more accurate solutions, because they reduce the input dimensionality and they can eliminate irrelevant or redundant features. Entropy based feature selection algorithms, such as MRMR (Minimum Redundancy Maximum Relevance, [1]) and FCBF (Fast Correlation-Based Filter, [2]) are preferred feature selection methods because they are very fast and produce sets of features that result in quite accurate classifiers. Besides accuracy, stability is another measure of goodness for a feature selection algorithm. A feature selection algorithm is said to be stable if changes in the identity of data points available for feature selection still result in the same or similar sets of features. In this study, we first developed a new stability measurement and performed accuracy and stability measurements of MRMR when it is used on different data sets. We found out that, the two feature selection methods within MRMR, MID and MIQ result in features with similar accuracy. On the other hand, MID results in more stable feature sets than MIQ and therefore should be preferred over MIQ, especially for small number of available samples.