Enriching predictive models using graph embeddings

Abstract

Today, artificial intelligence and machine learning models are developing rapidly and their usage areas are increasing in proportion to this speed. These developments have led to an increase in forecasting models in various industries. In the digitalizing, changing, and rapidly increasing data world, the spread of these solutions can be seen as a result of the moves made by companies to seize their competitive power and increase their profits. Especially with the increase in the amount of data, the need of the prediction models has also increased. Predictive models are used in various fields. For example, in financial markets, forecasting models are used to predict future price movements. Also, in the healthcare industry, predictive models are used in the diagnosis and treatment of diseases. In a graph structure such as social networks, prediction models can be used to predict users' interests. Especially in the field of e-commerce, where social networks are seen a lot, data is often represented as a graph structure. Graph embeddings can be used in a variety of prediction models. For example, when building a prediction model, we can improve the performance of the model by using graph embeddings. This helps the model better understand the relationships in the graph structure so it can make more accurate predictions. Also, graph embeddings can be used to predict relationships between nodes in a graph structure. For example, in a graph structure such as a social network, graph embeddings can be used to predict friend relationships between users. Demand forecasting systems and recommendation models are two different types of forecasting models. Demand forecasting systems are the accurate calculation of the future demand for a product. Graphs can be useful demand forecasting for several reasons. For example, demand prediction often involves working with sparse data, where there are many possible products or customers but only a small subset of them have any observed demand. Graphs can help to handle this sparsity by leveraging the relationships between entities to infer demand for products or customers that have not been directly observed. Graphs can be also used to handle the temporal dynamics of demand, such as how demand changes over time. Graphs can be very helpful to capture the compex interactions between different factors, such as customer preferences. Recommendation systems, on the other hand, are softwares used to make recommendations based on the interests of users. For example, in a music player app, by reviewing the songs a user has listened to and the artists they've listened to, other songs can be suggested that match that user's interests. Since recommendation systems generally work by finding the user or product similarities in data sets, using graph embeddings for performance improvement in recommendation systems can significantly improve performance. As a result, data can be easily represented in graph structures in demand forecasting and recommendation systems. Graph embeddings from these graphs are used as an important method in demand forecasting systems and recommendation systems and can help improve the performance of forecasting models in these areas. This thesis aims to use graph structures in demand forecasting and recommendation systems and to improve models with embeddings obtained from these graphs. This study generally consists of 2 parts: The use of graph embeddings in demand forecasting and the use of graph embeddings in recommendation systems. In demand forecasting, which is the first stage, although the success of cutting-edge machine learning and deep learning models in demand forecasting has come to the fore according to recent research, the usage of datasets that are enriched using graph-based feature representations to improve demand forecasting models, is still rare. Therefore, in the demand forecasting stage, a model that predicts demand using graph embeddings is proposed. Unlike most existing methods, sales information data is used to extract various relationships between products, and these various relationships are used to create graphs. Five different embeddings are evaluated to reflect the different relationships between the products. These five different embeddings are obtained using five different graphs created using five different relationships. In order to obtain product embeddings, two different graph embedding methods named Node2Vec and GraphSAGE were preferred. After the product embeddings are obtained, Long short-term memory (LSTM) and Extreme gradient boosting (XGBoost) models are tried to perform demand forecasting. The Mean absolute error (MAE) metric was preferred to test the accuracy of the predicted values and the model was tested using a publicly available retail sales dataset. By running both models with different parameters, the best parameters were found and the results were obtained by using the best parameters while taking the final results of the models. According to the results, the use of graph embeddings increased the prediction success in both models. The prominent model came into prominence as the model formed by the Node2Vec graph embedding method and the LSTM method with the value of MAE=2.98. Another study is the use of graph embeddings in recommendation systems. The model proposed in this study obtains graph embeddings for each product using the product title information. At this stage, a model used in natural language processing called BERT was used. With the proposed model, one embedding was created for each product using product titles, and category (topic) information was obtained for each product using the embeddings from the BERTopic model. This category information obtained is then used in the the next stage and enriches the user-product bipartite graph by transforming it into a heterogeneous user-product-topic tripartite graph. Unlike most existing methods, categories are created using only product titles as attributes, making it an easy-to-implement methodology that can be applied in real-world systems even if items don't have detailed item descriptions or comments. In addition, it is aimed to solve the sparsity problem in recommendation systems by learning the node embeddings using the meta-path-based Metapath2Vec algorithm to explore the resulting tripartite graph. Two different categories of the Amazon dataset were used to observe the performance of the proposed model, and the results showed that the model outperformed even the closest basic result by 3.8\% when evaluated for both datasets according to hit raito. Further analysis confirms that incorporating category information into the graph structure provides an increase in recommendation accuracy, especially when recommending long-tail items. As a result, the proposed model has been evaluated using small versions of datasets and versions where we call cold-start products with little relevance, and the proposed model outperformed the base models for both hit rate and NDCG.