2016 Results

Amendment

Note that the results originally announced at AAAI 2016 were erroneous.  In particular, some additional matches that were run to separate the top four agents, seeds 100 and higher, were biased as all these matches used the same sequence of cards.  This bias invalidates the previously posted results of the Bankroll Instant Runoff competition.  We gratefully thank Vahe Hagopian who brought this to our attention.  

To correct this error, additional matches between the top four, and subsequently the top two agents, were run.  The log files, crosstable and ordering listed below have been corrected.

Results

The results of the tenth Annual Computer Poker Competition were announced February 13th at the Thirtieth Conference on Artificial Intelligence in Phoenix during the Computer Poker and Imperfect Information Games Workshop.

This year, the competition used two player no-limit Hold'em as its testbed.  There was insufficient interest in the 3 player Kuhn poker event and the limit Hold'em events were temporarily removed until they can be revamped.  No-limit Texas Hold'em had an event using the total-bankroll winner determination rule, to encourage learning techniques as well as an instant run-off event that encourages defensive play, as in traditional game theoretic techniques.

About the Results

The rules for this year's competition called for matches of duplicate poker to help mitigate the effects of "luck" and more accurately determine this year's winner. In total, about 15 million hands of poker were played.

In all of the PDF result files, the top table is a crosstable including all combinations of matches between players in the given variant of poker.  Crosstable entries are the expected winnings, in thousandths of a big blind per hand, for the row player.  The second value is the 95% confidence interval around the mean.  Cells are shaded bright green/red if the row player won/lost by a statistically significant margin (95% confidence.) Cells are shaded light green/red if the row player won/lost by a statistically insignificant margin. If a cell is grey no data is available for that combination of players (e.g., matches of an agent against itself or an agent entered exclusively under a different winner determination rule).

The raw, uncapped result files have a mixture of instant run-off and total bankroll agents, with no further processing or information.  For the total bankroll result files, the second table gives the probability that the row player has a higher total bankroll than the column player.  The SUM column is simply the sum of the values in the row, and is NOT used in winner determination.  For the instant run-off results, the second table gives, for each round of the run-off, the SUM statistic describing the total bankroll performance of the remaining players.

Statistical confidence was not possible for all results. We have noted all such decisions with a '*'. Please see the section discussing the statistical analysis used to generate the results for additional details.