The recent trend of using machine learning algorithms to guide poker agents (Li et al., 2020 Brown et al., 2019 Zarick et al., 2020 Moravčık et al., 2017) has shown promise, but still suffers from a lack of interpretability for humans. This is a serious drawback, and one which will severely limit the degree to which these methods will be accepted by regulators and the society at large. While this is an impressive milestone, and certainly a critical step towards the ultimately applying control towards real life problems (i.e., non-recreational games), unfortunately to-date all proposed methods have suffered from a lack of interpretability. Nonetheless, in recent years we have witnessed several breakthroughs which have yielded poker bots capable of achieving super-human performance (Brown & Sandholm, 2018) and capable of beating human professionals (Moravčık et al.

In contrast, progress in the domain of imperfect-information games, such as Poker, where the players hold disjoint public/private information sets, has proven to be notoriously difficult. Over the past two decades, reinforcement learning has yielded phenomenal successes in the domain of perfect-information games: it has produced world-class bots capable of outperforming even the strongest human competitors in games such as Chess and GO (Silver et al., 2018). Most exciting of all, the resulting poker bot is highly interpretable, allowing humans to learn from the novel strategies it discovers. Through experiments against Slumbot, the winner of the most recent Annual Computer Poker Competition, we demonstrate that our approach yields a HUNL Poker agent that is capable of beating the Slumbot. Second, we make use of globally optimal decision trees, paired with a counterfactual regret minimization (CFR) self-play algorithm, to train our poker bot which produces an entirely interpretable agent. Namely, first we propose a novel, compact, and easy-to-understand game-state feature representation for Heads-up No-limit (HUNL) Poker.

In this paper, we present advances on both fronts. This lack of interpretability has two main sources: first, the use of an uninterpretable feature representation, and second, the use of black box methods such as neural networks, for the fitting procedure. We address the problem of interpretability in iterative game solving for imperfect-information games such as poker.

Machine learning for playing poker