Usage of Recurrent Neural Networks (RNNs) in Games

Advantages of RNN

This approach is very attractive for modeling players’ behaviors within game environments, especially when trying to deduce strategies or predict the next action. This is due to the technique outperforming previous work mentioned in this chapter, as well as alleviating the expense of using expert knowledge to manually define planning domains and probability distributions. Although the approaches described above don’t fully eliminate all the manual work, they reduce some aspects of it. Bisson et al. (2015) increase accuracy relative to nonmachine-learning-based techniques while automatically determining probabilities and vectorized feature representations for their HTN hypotheses. Using RNN, Min et al. (2016) avoided the need for a plan library and manually transformed the world state to extract important features. Instead, they allowed the LSTM to automatically determine the relevant actions and world state features.

Drawbacks of RNN

However, in spite of these advantages, there are some disadvantages that make the use of RNNs for games problematic. Specifically, as is the case with many machine learning methods, a considerable amount of data is required to train the model. Additionally, the approach lacks an explanation or understanding of the learned strategies or of how the sequences of action data pertain to the goals.

For instance, Bisson et al. (2015) provide no clear method for manually interpreting or inspecting the feature vectors or plan to determine why certain replays lead the algorithm to conclude that particular hypotheses are more likely than others. Min et al. (2016) predict goals, but they don't describe a method for expanding their system to predict full plans.

Although other researchers are working on improving the interpretability of neural networks, they often rely on visual information to depict the networks’ decisions (Olah et al., 2018), which may not apply naturally to plan recognition problems. If you found cases where the RNN-based plan recognition algorithms failed to predict the correct goals or plans, it would be difficult to tweak them to fix those specific cases without gathering additional data and retraining the algorithm, which might introduce other unforeseen problems. For this reason, many game designers currently prefer more interpretable approaches, except in cases where they can review the output manually before it’s shown to players.

In other words, while the technique seems to be more feasible to use for commercial games and provides great results compared to other methods discussed in this chapter, it may lack the explainability that may be needed for different stakeholders.

Applying in games

RNNs have been used in game data science for some time. Since RNNs can remember a long sequence of information, it performs better in handling time-series data than other techniques. RNN-based techniques have also been used for procedural content generation as well as AI agent generation (Summerville and Mateas, 2016; Summerville et al., 2016; Lample and Chaplot, 2017). To give an example of using RNN in-game data science, we will focus on the work of Guitart et al. (2019), who devised a method based on RNN to predict churn. Their work won the IEEE CIG 2017 game data mining competition. An example of churn prediction was discussed above using HMM. Here, we'll discuss RNN as an alternative approach.

As with the study discussed above, the goal of this work is to predict if a player will stop playing the game as well as their remaining game lifetime. This was divided into two problems:

1. Predicting if a player would stop playing or not

2. Predicting the time when the player would stop playing

The data used for this work is taken from a multiplayer online game called Blade and Soul (NCSoft, 2012). In this work, tree-based ensemble learning and RNN, specifically LSTM, have been used to solve the two problems.