34

I'm interested in building a Texas Hold 'Em AI engine in Java. This is a long term project, one in which I plan to invest at least two years. I'm still at college, haven't build anything ambitious yet and wanting to tackle a problem that will hold my interest in the long term. I'm new to the field of AI. From my data structures class at college, I know basic building blocks like BFS and DFS, backtracking, DP, trees, graphs, etc. I'm learning regex, studying for the SCJP and the SCJD and I'll shortly take a (dense) statistics course.

Questions:

-Where do I get started? What books should I pick? What kind of AI do poker playing programs run on? What open source project can I take a page from? Any good AI resources in Java? I'm interested in learning Lisp as well, is Jatha good?

1
  • 12
    have you succeeded in developing anything ? It's been 2+ years... May 3, 2012 at 7:25

9 Answers 9

32

The following may prove useful:

6

Poker AI's are notoriously difficult to get right because humans bet unpredictably. It's usually broken into two parts.

1) Calculate the odds of your hand being the winner.

2) Formulate betting strategy based on 1.

I'd recommend starting with lots of statistics reading for part 1. It seems easy at first blush, but it's actually very complicated (and getting it wrong will doom your AI). Then move on to genetic algorithms for part 2. Betting strategies are mostly genetic algorithms. They adjust themselves based on past success and failures + some randomization so as not to become predictable.

3

I wrote a Texas Hold'em Video Poker engine in Java

This code is a core engine for Texas Hold'em without views and others

http://github.com/phstc/javapokertexasholdem

2

Also, letting genetic algorithm adjust the weights of neural network, which determines the decision logic. This approach is very suitable for poker AI.

I made my own AI like this. At first, I created ~1000 players, who didn't know how to play the game at all. Based on their initial luck during the hands, their fitness was weighted and new generation created. New "brains" were playing better than previous generation.

Eventually, the best individuals played very good.

4
  • 4
    Did you publish your results anywhere? May 7, 2009 at 11:08
  • When you say the best played very well, do you mean against the other AIs or against humans? I would think a genetic algorithm would perform poorly. Poker is about understanding your opponents, not odds. Unless your genetic algorithm included information about distance from the button when betting I would find your result suspect. May 13, 2009 at 19:08
  • 1
    You didn't get it. GA didn't perform at all, the neural networks did. GA was just to evolve the nets (adjusting weights) based to their success. I played against the best individual and it was pretty good. There was no detectable pattern in it's logic. I also datamined 500k hands from one poker client, and made models from few of the best players (with neural networks). The best individual did beat these as well. The average BB was around 1.5 after one million hands.
    – nhaa123
    May 14, 2009 at 9:10
  • 2
    do you have any more details about this?
    – f0ster
    Oct 24, 2011 at 21:20
2

As already recommended, the book Theory of Poker is a truly invaluable source of information for playing the game as well as for building an AI. You should probably buy it as it does not cost that much.

University of Alberta resarch group does the state-of-the-art at the moment, though they have stiff competition emerging every now and then. (Not all poker bots and AI research in the field is public because of the temptation to use one's results in internet poker, though that's forbidden.)

First you should decide what sort of poker are you going to tackle first. two player hold'em is pretty much solved, though the best humans still put up a real fight with the best AI's available. The AI has the main advantages over humans by having an unlimited flawless memory of past hands, flawless analysis of the patterns based on that and as they are machines, they don't tilt like almost all humans occasionally do.

Fixed Hold'em is probably the easiest to crack, so you might want to start with 1-1 fixed hold'em and then decide what you want to do next.

Here are some aspects which change the correct strategy (and your AI):

  • A cash game is different from a tournament -The number of players makes the decisions different.
  • Hold'em is not the only poker. Omaha, Stud and others exist and are widely played.
  • Fixed Limit is different from Pot Limit, which is different from No Limit.
  • To beat the best you need to cover a lot of very subtle things the best players think about when they play. To beat a low-stakes amateur game, none of these things count.

If you decide to go for No Limit Hold'em, you might want to check out three-book series Harrington on Hold'em and a book No Limit Poker - Theory and Practice. Having read quite a many books on poker, I can say these books combined with the Theory of Poker are quite enough.

2

I'm not sure which exact game you are interested in, but the typical approach is to create a much smaller abstract version of the game, solving that smaller game, and then mapping real game situations back to the abstract game to generate advice. Most of the academic papers skip over the details of this process in favor of presenting results about convergence, exploitation, and competition results.

However, there are some publicly available code bases which present a complete implementation. One of the best ones is Fell Omen:

http://www.deducer.org/pmwiki/pmwiki.php?n=Main.ArtificialIntelligencePoker

This is a basic complete strategy bot that uses fictitious play to optimize the strategy for the abstract game. It's a good starting point because it is fairly straightforward, complete, and represents a good presentation of the abstract game approach.

If you are interested in developing poker AI, I would suggest reading everything from 2007 and on from the UA poker group and Tuomas Sandholm's students:

http://www.cs.cmu.edu/~sandholm/

http://poker.cs.ualberta.ca/publications.html

1

I wrote a Hold'Em AI in my undergrad. It wasn't particularly advanced, I used a Q-Value machine that traversed a number of states and updated Q values for each state.

I found the University of Alberta's AI Poker project an invaluable source of info for avoiding pitfalls.

As one poster above states, the first step is to nail in a couple of determinable poker rules - one-on-one poker can be developed programatically.

One pitfall I fell into was not building in reconfigurability early on. For instance being able to switch the grade of learning/playing.

I would be interested to hear how you get on drop me a mail stevekeogh at gmail.com

0

Just to add to the links above, one of the important things to implement would be http://en.wikipedia.org/wiki/Kelly_criterion which will help figure out the optimal size of bets given the expected odds in a series of bets. With humans there could be errors in judgement of odds, but if your AI program can spit the some expected normalized odds based on whatever the algorithm then this bet-sizing technique which balances both risk and reward for the advantage gambler would be a good cheap solution.

-1

One interesting result I've heard is that if you restrict the betting options to fold, check and all-in, you can write an AI that wins one-on-ones with probability at least 49%, and 49.5% if it's (IIRC) not going first.

I don't know that this AI is easier to write that one which knows how much to bet, but it's food for thought: choosing amounts to bet only accounts for 1.5% of the probability of winning.

Not the answer you're looking for? Browse other questions tagged or ask your own question.