Mh-nais: a multi-head attention extension of the neural attentive item similarity model

Abstract

Personalized recommendation systems are widely used in many areas such as online publishing platforms, e-commerce and social media. These systems determine the interests of users based on their interactions and increase the user experience by making appropriate recommendations. Collaborative filtering method is one of the basic techniques of recommendation systems. However, traditional methods are limited in creating user and product representations and are not sensitive enough to complex user preferences. In recent years, the inclusion of attention mechanisms in recommendation systems has made significant progress in overcoming these limitations. The Neural Attentive Item Similarity (NAIS) model is one of the studies that increases recommendation accuracy by assigning different weights to the items that the user has interacted with in the past. However, since the NAIS model uses only a single-head attention structure, it may be insufficient to fully reflect the diversity in user behavior. In this study, the attention mechanism of the NAIS model is extended with a multi-head attention structure and a new model called MH-NAIS is proposed. By using multiple attention heads at the same time, MH-NAIS can learn from various representation subspaces in parallel. In this model, each attention head learns the relationship between the target element and the elements the user interacts with a different representation. This allows it to create more expressive and strong user profiles. Thus, it is possible to increase the recommendation accuracy by modeling the various interests and preferences of the users more effectively. The model was implemented using TensorFlow and the success of this model was evaluated on the MovieLens 1M dataset and the obtained results were compared with the previous NAIS and FISM models. Key performance metrics, including Hit Ratio at rank 10 (HR@10) and Normalized Discounted Cumulative Gain (NDCG@10), were used to assess the effectiveness of the proposed model. Experimental studies showed that the Multihead approach outperforms the baseline NAIS model, with up to 1.5% improvement in HR@10 and 3.93% in NDCG@10. The results show that multi-head attention significantly improves recommendation accuracy and robustness. Additionally, the various attention heads help better understand how users interact with items. In addition, although the training time of the proposed model is longer than the classical NAIS model, this increase was found to be balanced by the improvements in the quality of the proposed model. In summary, this study makes a significant contribution to the recommender systems by introducing a model that utilizes multi-head attention for its strong representational capabilities. The results indicate that incorporating such mechanisms into item-based collaborative filtering models, like NAIS, can significantly increase their predictive ability and performance. In the future, this model has a high potential to be used in different datasets, hybrid recommendation systems and in solving cold-start problems.