Accelerating classification in large-scale and imbalanced datasets: A hybrid ann approach

Abstract

Purpose This study proposes a novel hybrid artificial neural network (H-ANN) framework, inspired by reinforcement learning (RL), to proactively detect Internet connection speed problems using enriched datasets from multiple sources of an Internet service provider. Design/methodology/approach The problem is challenging due to the high dimensionality, unbalanced class distribution and continuous influx of new data. To address these issues, the proposed hybrid framework integrates supervised learning methods – radial basis function network (RBFN) and multi-layer perceptron (MLP) – with the unsupervised self-organizing map (SOM). RL is employed to accelerate learning, reduce feature and instance space complexity and improve the detection of underrepresented classes. The framework is first validated on benchmark open-source datasets and subsequently applied to real-world company databases combining network, business and customer information. Findings The results demonstrate that the proposed H-ANN significantly improves both classification accuracy and computational efficiency compared to conventional machine learning approaches. Importantly, the framework enables the early identification of slow Internet connections before customers submit complaints, allowing the service provider to take proactive measures. Originality/value The proposed H-ANN framework not only enables the early identification of slow Internet connections before customers submit complaints – allowing service providers to take proactive measures – but also offers a generalizable solution for large-scale, imbalanced and dynamic data classification problems across diverse domains.