Since chess computer Deep Blue defeated world chess champion Garry Kasparov in 1997, artificial intelligence has held increasing sway over humans in a handful of 'adversarial' games - those in which human interaction plays a limited role. Now, however, a group of researchers have revealed a new AI that's attempting to expand the pool of games that a computer can beat you at.
In a paper published this week, researchers unveiled Cicero, an AI trained to win games of Diplomacy, a seven-player board game in which "each turn, all players engage in [...] free-form dialogue with the others during a negotiation period" before taking an action. That discussion phase is what sets Cicero's efforts apart from other AI.
The paper states that "almost all prior AI breakthroughs" have been in "two-player zero-sum" games, in which gaining an advantage for oneself puts the other player at a direct disadvantage. In those games - Chess, StarCraft, Go, and Poker - the AI can learn optimal strategy by playing against itself in a pattern known as 'self-play'. Eventually, it will come up with an approach that can't be beaten in a balanced game. In these examples, the complexity of the game itself isn't important; what matters is that communication isn't a central game mechanic, and that each action strives to set another player back in their goal.
That's not true of Diplomacy, a game in which conversation between players is important (if not entirely crucial), and in which making a gain does not necessarily harm an opponent. Here, self-play "produced uninterpretable language." That was a major obstacle to overcome, as anonymity was key to a fair experiment. Communication between players had to be grounded in the state of the game, or events that had already occurred, and if Cicero slipped up, the likelihood was that it would be found out due to its inability to explain its mistake.
Even more important, however, was the ability to build trust with fellow players. Theoretically, that concept would be alien to Cicero, but to succeed it would need to establish "an ability to reason about the beliefs, goals, and intentions of others" as well as "an ability to persuade and build relationships through dialogue."
To establish Cicero, researchers took a dataset of more than 40,000 dialogue-driven games of Diplomacy from an online version of the game. A base dialogue model was then trained on the Diplomacy chat logs, and then trained to predict messages based on an array of game data. Eventually, Cicero was trained to "exploit" the information in a message when deciding on its next action, while also reasoning what other players might be attempting to do.
