Analyzing player engagement in western action role playing games using user reviews and achievements

Abstract

User reviews are one of the leading factors of judging player engagement and popularity of digital video games. They not only provide sentimental feedback but also give insight on what the player likes or dislikes about a product. However, it is confusing to see this segment not reflected in tools of player engagement such as game achievements. Game achievements provide objective data related to the successes of a player in terms of engagement. We believe we can compare these two user outputs to create a correlation between what players say and what players do in relation to player engagement. A sentiment analysis can be done using the frequency of words used in reviews and review recommendation to understand why players use the words they use. In this thesis we will be focusing on western action-based role-playing games, comparing Steam user reviews with Steam achievements by categorizing each based-on type, and consistency. Game achievements will be categorized based on player experience, engagement, and enjoyment aspects. User reviews will be ran through a stemmed word frequency query, manually selected and divided between predefined categories. Data gathered from these categorizations, such as word count and completion rate was then compared with reviewer recommendations to analyze the sentiment regarding each defined player engagement aspect. Furthermore, k-nearest neighbors algorithm (k-NN), a non-parametric classification model, was used to create review recommendation level clusters to analyze and player reviews for other games of this genre. In conclusion, we have found that a correlation between frequency of words used by reviewers and average achievement unlock rates of games can be made. We have also noticed correct applications of different aspects of game engagement may have an impact on the other aspects, as a game with engaging gameplay might lead to lower rates of game completion while a poor implementation of exploration mechanics that lack engagement can lead to increased game completion rates. We have also found the correlation between our two main datasets can be used to create a classification model to analyze review sentiment rates of other games moving forward. Going forward, more research on different video game titles or genres might be necessary for understanding why increased success in one aspect of engagement leads to decreased interest on other fields.