A novel approach for time series forecast combinations based on multi-criteria decision making (MCDM) methods

Abstract

Forecast combinations in time series are quantitative forecasting methods that aim to obtain models with higher performance results in terms of accuracy by integrating the strengths of different time series forecasting models. This study aims to develop a new multi-dimensional and objective approach using multi-criteria decision-making (MCDM) methods that outperform both single forecasting models and some of the most widely accepted forecasting combination models in the literature. In this context, a new combination model is developed in which the weights of the forecast models are assigned by using some hybrid MCDM approaches. First, the model proposition of CRITIC-TOPSIS-based predictor assignment was applied to two different datasets. The reason to test the same model on two different datasets is to understand the efficacy of the proposed method for time series having different underlying patterns, characteristics, sizes, and frequencies. The initial dataset consists of daily data from the industry domain, sourced from the M4 competition file which is named D2035, and is publicly available as open data on GitHub. It includes irregular patterns, exhibits high-frequency noise, and volatility as well as is large in size. Then, the same proposed model was applied to a monthly dataset of deliveries of natural gas to electric power users (MMcf) for the United States of America (EIA's dataset), which includes more regular and various patterns, exhibits lower-frequency noise and volatility, as well as it is smaller in size. After preliminary preparations and statistical tests, the datasets changed to conform to the specifications of the statistical application (if necessary) and were split into 3 subsets named train, validation, and test. All individual forecast models were trained on train sets and forecasted on validation and test set horizons. The selection of predictors (so, the components of combination forecast) is handled by visual inspection of the graphs over validation and test horizon. The decision matrix of predictors versus performance metrics was created as a next step. The performance metric results from the validation sets are used for each predictor to build this decision matrix. Using a hybrid MCDM approach, where the performance metric weights were assigned through the CRITIC method and the TOPSIS model first, was applied based on these weights, the closeness of each predictor to the ideal solution was determined. The normalized ratio corresponding to each predictor's closeness to the ideal solution will represent the weight within the new proposed combination model. The relative test performance results derived from both datasets are compared with traditional single forecasting models, machine learning models, and several widely used time series forecast combination models from the literature, as well as with each other. The results indicate that the proposed CRITIC-TOPSIS hybrid MCDM-based forecast combination model outperformed all individual and existing combination models on natural gas delivery to USA electric power consumers (MMcf). On the other hand, the D2025 dataset did not yield performance results as outstanding as the MMcf dataset. However, it still provided promising outcomes with notably positive results based on certain performance metrics. In this context, it can be stated that the proposed and implemented model performs better on datasets with more defined patterns, lower noise, and reduced volatility, rather than those with high randomness, high-frequency noise, irregularity, and ambiguous patterns. Additionally, the proposed model yielded highly effective results compared to its counterparts. After superior outcomes of the proposed CRITIC-TOPSIS-based forecast combination model, especially on the natural gas delivery to USA electric power consumers dataset (EIA), it is decided to apply the same combination methodology on the same dataset with the CILOS-TOPSIS hybrid MCDM approach this time. Testing the proposed method with both the CRITIC and CILOS approaches enables a comparative analysis of how differing weight assignments for accuracy measurements based on their relative influence will affect the success of the combination forecast. The findings reveal that the proposed CILOS-TOPSIS hybrid MCDM-based forecast combination achieved small enhancements compared to the CRITIC-TOPSIS-based approach. Additionally, another hybrid MCDM approach model built under the umbrella of the proposed model for forecast combination, the CILOS-ARAS-based model, was tested on EIA's dataset to evaluate potential improvements arising from modifications to the performance evaluation of the decision matrix. This approach facilitated the assessment of the utility function value, as opposed to relying on the closeness to the ideal solution using a TOPSIS-based evaluation when determining weights for the component forecast models. All MCDM-based models created under the proposed framework are compared with each other and with other existing models in the literature.