Player Matching, Mastery, and Engagement in Skill-Based Games
Earlier this month, Waterloo University hosted its first (I think) Gamification conference, called Gamification 2013. The conference was a great opportunity to publish some of the recent work I’ve done for Skillz (together with my good friend, Naor Brown, from Harvard University).
The paper explores (as the title suggests) the relationship between player matching, mastery, and engagement in skill-based games (there’s a link at the end, in case you want to download the original paper). In the paper, we make the following claim:
Player matching that is fair (= players are on the same skill level), gives players a sense of progress (= on the path to mastery), which in turn keeps them engaged.
For example, if you match a strong player with a weak player, the strong player will get bored (not enough challenge), whereas, the weak player will get frustrated (to much challenge). Either way, the likelihood that both players will keep playing (= engaged) is very small.
Challenge vs. Skill
The approach we proposed for the player matching problem was an algorithm based on the Elo rating system, a method for calculating the relative skill levels of players in competitor-versus-competitor games such as chess, named after its creator Arpad Elo, a Hungarian-born American physics professor. Intuitively, our approach says that a higher rated player gains fewer rating points by beating a lower rated player, and a lower rated player gains more rating points by beating a higher rated player. Over time, players reach their relative skill level. We use this approach to model players’ skill and relative rankings, as well as, modify ratings once players have played against one another, resulting in fair matches between players with the same skill level.
Although the focus of this research was skill-based games, we believe our approach (and claim) has implications for Gamification, that is, to increase fairness in gamification by matching users and tasks based on skill level. For example, we know that if places on a leaderboard are calculated relative to first place, then the leaderboard in not balanced which may cause occasional users to stop playing, and new players to not play at all. Our approach can segment users into groups based on their ratings calculated from points awarded, badges earned, missions completed, etc. These weighted groups can then be fitted with custom leaderboards that display a player’s place relative to others in the group. Furthermore, users in gamified applications are required to complete various tasks. Our approach can be used to ensure that users are on the path to mastery (= engaged) by matching them with tasks that are within their skill level.
Takeaway message:
Make sure that user-to-task matching within your gamified environment follows the Goldilocks principle, i.e., if a task is too difficult, a user will likely get discouraged and quit. Moreover, if a task is too easy, a user will not feel challenged and quit. A fair user-to-task match ensures that a user remains on the path to mastery and, therefore, engaged.
Download the full paper