Current MS Students/Dan Langlois

From CSWiki

Predictable Player Poker Bot Project

[edit] What I am planning to accomplish

[edit] Brief Project specification

Poker is a very mathematical game. I intend to program a Poker bot which has math ability, but does not attempt, and will not attempt to defend itself against, deceptive plays. Its goal will be to make the best possible play every time.

I see the the problem of assessing how strong the bot's hand is as the most critical subroutine needed. What is generally referred to as 'hand strength', is presently having a hand. In Poker strategy discussions, this idea is combined with the tool of 'pot odds', which is evaluating the cost to see one more card against the potential for improvement. However, my Poker bot will have a better tool--a detailed hand odds calculator. It will genuinely compute the long-term average outcome of given scenarios (a closely related concept is sometimes referred to as 'showdown odds', and is difficult for a person to calculate on the fly).

Traditionally, a big part of poker has been that it is a game of deception and of inference. Many good poker players are said to win not because they have a perfect knowledge of the odds, but because they can read the other players. Conversely, they can also deceive the other players playing. So, in a sense, this Poker bot will not have advanced beyond the fundamental mechanics of poker playing, and will have only begun to strategize on optimizing gains and minimizing losses. However, the basis for every decision it will make when playing will be mathematical expectation. And, by being predictable and transparently logical, the Poker bot will isolate and exemplify probabilistic knowledge and risk assessment in Poker strategy.

[edit] High level architecture

Game Modeler
This is the highest-level container for the other modules. It can start up a game, with some parameters, the most basic of which is how many players will be in the game (and how much money they start with).

Dealer
The dealer deals out the cards randomly, from a 52-card deck.

Poker bot
Note that I figure the poker bot needs to be able to play itself for testing purposes, so a variable number of players can be modeled. Each receives 2 hole cards, pre-flop, and examines the community cards and makes betting decisions during the game. The only decisions the poker bot needs to be able to make are whether to bet, call, or fold. From the most abstract point of view, it makes these decisions based on what cards it has in its hand, and on the betting patterns of the other players. However, it will

variables, including:
-how many players have raised during the round
-how many players are in the game
-how much money is in the pot, is relevant for hands which are not simply good enough to bet/raise with.

Ranked Hands Lists Generator
The ranked hands lists generator will accept the community cards as input and return a ranked list of 2-card hands. This list will be used by the poker bot to assess the 'objective' strength of its hand, which is how it will decide on bets/calls/folds during each betting round. I am currently thinking that the pre-flop list will not be generated dynamically, as it will be more computationally intensive to generate this one. However, coincidentally, the pre-flop card hands list is the one that is completely reusable, as no input (of community cards) is required.

[edit] Why this is academically interesting

The project will use statistical modeling, and some game theory. Naturally this general area has also interested researchers around the world. For years, games researchers have used chess, checkers and other board games as a testbed for machine intelligence research (researchers have spent a lot of time and effort on board games such as chess and checkers). In chess you already know all the moves possible for yourself and your opponent. But poker has seemed to be a different beast all together. However, the American Association for Artificial Intelligence (AAAI) has held its first Computer Poker Competition at the annual AAAI conference. And, at the U of A, the Computer Poker Research Group, led by Jonathan Schaeffer, is part of the The University of Alberta GAMES Group which examines game theory across the board, so to speak.

There are some poker bots around, but I can't simply download my notion of a 'predictable & transparent' poker bot anywhere. Such a demo poker bot would aid considerably, I think, in wading through poker strategy discussions, including the literature on Poker AI. And, with the 'normal' strategy rigorously implemented, some quite counterintuitive insights into poker will be conveniently demonstrable, such as, that a betting hand is, from a one-dimensional mathematical point of view, a hand that sorts out as simply a better hand than a calling hand. Also, that 'weak' play and 'deceptive' play are not to be distinguished mathematically.

[edit] Previous work

[edit] Platforms to be used and what I will add to each

The UofA has released some of the Java code from the poker project, most notably the Hand Evaluator (Java). It can get the best 5 card poker hand from a 5 to 7 card hand. I am certainly going to need to evaluate hands, but in fact, I will need to iterate through and evaluate them, doing an exhaustive search. Actually, I am assuming that I can do 30 or 20 or, hopefully, 10 million hand evaluations per second, and if I can't, well, maybe Java isn't the language for me to use. poker-eval is a C library to evaluate poker hands (on sourceforge).

I'm leaning towards using the Plug-In API at poker-academy.com:
http://forums.poker-academy.com/viewforum.php?f=3

[edit] Available platforms I am not using and why

Interesting, and open source, is Real Poker 0.2.8, on SourceForge, but it trains robot players against each other.

Also, there are some online poker sites offering analysis software for poker players. The most popular of these, is Sit and Go Shark. It is billed as an 'advice engine'. The advice comes from a 'HUGE' database of text. However, this 'advice engine' attempts to address issues like 'table momentum', which can mean how fast, slow, tight, loose, aggressive, or passive a game is going, and 'betting patterns', for example, if a player loses half of his stack in a single hand, he's much more likely to 'tilt' and make a stupid move within the next few hands.

Also, there are tools, or simulation utilities (but apparently nothing open source), to help calculate 'pot odds' in real time and especially (from my point of view), software which calculates the probability of winning, as well as calculating the equity for specific hands in a direct confrontation, while considering the community cards and the mucked cards. That is to say, there is software that can be used to compare hands. For example, the 'Equilator'. There is also an applet online at http://www.pokerware.org/ which does this. The two approaches for making this calculation that I have seen discussed are combinatorics and Monte Carlo. Monte Carlo is the ubiquitous little beast of burden in Bayesian statistics. Combinatorics is much more straightforward and more accurate, and involves generating all possible boards at each stage, and, for each board, generating all possible opponent hands. My Poker bot will have a closely related use for combinatorics. The idea will not be simply to compare hands, but the underlying algorithm seems extremely simple.

Then there is Poker Academy Pro, which is a training package (a hyper-advanced goal), including heads-up specialist bots (which employ “dumb” situational strategies, some of which are pointed out). Its hand evaluator, with its wealth of poker odds calculations, gives me confidence in its in-game advisor. So, I am impressed--but not demoralized (see 'Why this is academically interesting').

[edit] Anticipated challenge(s)

I plan to program the Poker bot to play, but also to be able to give what its play would be, in a given situation.

I am not interested in the performance contribution of any particular strategy, so I do not intend examine the feasibility of a data accumulation or learning system to infer anything. A lot of potential data that could be accumulated will be ignored.

[edit] What I bring to this work

[edit] My relevant background and experience (Diary)

[edit] What I find interesting about this work

The relationship between mathematics and poker is interesting and murky. While some good material on poker mathematics has been developed, until now it's been scattered and hard to find. For all the qualities of many of books written about how to play poker, they are lacking in sophisticated mathematical and logical proof of the concepts they present.

My starting point is this paragraph from Chen and Ankerman’s Mathematics of Poker:
"It is reasonable to look at each poker decision as a two-part process of gathering information then synthesizing that information and choosing the right action. It is our contention that intuition has no place in the latter. Once we have a set of assumptions about the situation - how our opponent plays, what our cards are, pot size, etc., then finding the right action is a simple matter of calculating expectation for the various options and choosing the option that maximizes this.”

This book takes around 15 very narrow specific "toy games" (i.e. I can only make one bet, and you can only call or fold), and runs through some very dense, mathematical calculations