LLMs were told they could lie in Diplomacy. Here's who actually kept their promises. [D]

the stats are from the game of diplomacy.

diplomacy is basically a strategy game where you negotiate, form alliances, betray, and outmaneuver other players to expand your influence.

the games were played in multi-agent simulations, with different LLMs playing against each other under the same rules and conditions with human as an opponent too!

you can read about the methodology here