Improve Counterfactual Regret Minimization Agents Training by Setting Limitations ofNumbers of Steps in Games

Counterfactual Regret Minimization (CFR) has been one of the most famous algorithms to learn decent strategies of imperfect information games. Because CFR requires traversing the whole or part of game tree every iteration, it is infeasible to handle games with repetition where the game tree is not finite. In this paper, we introduce two abstraction techniques, one of which is to make the game tree finite and the other one is to reduce the size of game trees. Our experiments are conducted in an imperfect information card game called Cheat and we introduce the notion of “Health Points” a player has in each game to make the game length finite thus easier to handle. We utilize the information sets abstraction technique to speedup the training and evaluate how results from smaller games can improve training in larger ones. We also show Ordered Abstraction can help us increase the learning efficiency of specific agents.

Counterfactual Regret Minimization (CFR) has been one of the most famous algorithms to learn decent strategies of imperfect information games. Because CFR requires traversing the whole or part of game tree every iteration, it is infeasible to handle games with repetition where the game tree is not finite. In this paper, we introduce two abstraction techniques, one of which is to make the game tree finite and the other one is to reduce the size of game trees. Our experiments are conducted in an imperfect information card game called Cheat and we introduce the notion of “Health Points” a player has in each game to make the game length finite thus easier to handle. We utilize the information sets abstraction technique to speedup the training and evaluate how results from smaller games can improve training in larger ones. We also show Ordered Abstraction can help us increase the learning efficiency of specific agents.