Evolution of Cooperation
In this project we explore which strategies prove most successful in the repeated Prisoner's dilemma game. We run evolutionary simulations of a population where each individual plays the prisoner's dilemma using one of several different strategies. Over multiple generations, players with a successful strategy persist while unsuccessful ones adapt, showing which approaches to cooperation survive in the long run. This project was inspired by "Evolution of Trust" by Nicky Case.
Repeated Prisoners' dilemma
The matrix below the prisoners' dilemma where, mutual cooperation yields a moderate reward (2, 2) for both players, while mutual cheating results in a poor outcome (1, 1). The dilemma emerges because cheating against a cooperator provides the highest individual payoff (3), but leaves the cooperator with nothing (0). This creates an incentive to cheat, even though mutual cooperation would benefit both parties more than mutual cheating.
| Cooperate | Cheat | |
|---|---|---|
| Cooperate | 2, 2 | 0, 3 |
| Cheat | 3, 0 | 1, 1 |
When this game is played repeatedly, however, new dynamics emerge. Players can remember what happened in previous rounds and adjust their behavior accordingly. Someone who always cooperates might start cheating after being betrayed too many times, while others copy what their opponent did in the last round. Some players forgive occasional betrayals but retaliate against persistent cheaters, and some may try to exploit overly forgiving opponents.
In our evolutionary simulation, the worst-performing players adapt by copying the strategies of the most successful ones. Over many generations, strategies that earn higher rewards persist while low-scoring approaches are forced to adapt, showing which collaborative behaviors can survive different environmental conditions.
Simulation Setup
The simulation features a population of 30 players with different strategy types. In each evolution round, every player plays 100 games against every other player using the prisoner's dilemma payoff matrix shown above. After each round, players accumulate scores that determine their fitness. The 3 lowest-scoring players adapt by copying the strategy of the highest-scoring player. To prevent strategies from disappearing permanently, one random strategy type is reintroduced each round.
Each player strategy is defined by three parameters: memory (how many rounds they remember), forgiveness (how much cheating they tolerate in the rounds they remember), and betrayal (probability of cheating). In these experiment we categorize the player in three colors:
- Green - These are forgiving players. If the opponent decides to cheat last round, these types of player are willing to forgive the other and still cooperate. A type of (0,0,0) will cooperate no matter what you do. However, sometimes forgiveness has a limit. A player of type (2,1,0) would forgive one times cheating in the last two games.
- Red - These are the bad types. They seem willing to cooperate at first, but they try to occasionaly cheat you. A player of type (0,0,0.3) cooperates 70% of the time but will try to cheat you 30% of the time. Such a sneaky player might be able to exploit the forgiveness of the green type.
- Blue - These are the cooperative, but non forgiving types. A player of type (100,0,0) will start to cooperate, but if you cheat on it once, then it will never cooperate again. These types don't let themself be exploited by the bad behaviour of the red types. The question remains, how cooperative is one that does not forgive?
- Cheater (0,0,1) - Always cheats regardless of opponent behavior
- Cooperator (0,0,0) - Always cooperates regardless of opponent behavior
- Grudger (100,0,0) - Cooperates until the opponent cheats once, then cheats forever
- TitForTat (1,0,0) - Copies the opponent's previous move, starting with cooperation
When mistakes occur, relationships can deteriorate unless players show forgiveness. We introduce forgiving variants:
- Forgiving TitForTat (2,1,0) - Remembers last two moves and forgives one cheat
- Forgiving Grudger (100,3,0) - Forgives up to three defections over the entire game
However, forgiveness can be exploited, so we also include occasional betrayal strategies:
- Sneaky Cooperator (0,0,0.1) - Generally cooperates but betrays 10% of the time
- Sneaky TitForTat (2,1,0.1) - Forgiving TitForTat that also betrays 10% of the time
To simulate real-world conditions, we add mistakes to the simulation. Each player has a small probability of making an error on any turn, causing them to act opposite to their intended strategy. This represents errors that occur in actual interactions.
Experiment 1: Four Basic Types
In this experiment we run the simulation for the four basic types: Cheater (0,0,1), Cooperator (0,0,0), Grudger (100,0,0) and TitForTat (1,0,0).
No Mistakes. TitForTat and Grudger become the dominant strategies. When they encounter each other, they collaborate effectively and don't let themselves be exploited by cheating players. Pure cooperators are quickly exploited by cheaters, while pure cheaters fail to cooperate and are also forced to adapt. This leads to a population that effective cooperates and achieves the highest possible average reward of 2.0 per game.
1% Mistake: Now we introduce a mistake probability of 1%. Mistakes can break cooperative relationships. A single accidental cheat can cause a Grudger to retaliate permanently, or make TitForTat players start cheating back and forth. In this simulation we can clearly see that the players are not collaborating well and average score of each player hovers around 1.7, instead of the average score of 2.0 of previous simulation.
10% Mistake:
When mistakes are too high effective cooperation completely breaks down and the Grudger strategy dominates.
This leads to a low average reward of almost 1.0.
Experiment 2: Forgiveness
To effectively cooperate when mistakes can be made, players might choose to forgive each other to prevent accidental mistakes from destroying collaborative relationships.
1% Mistake: In this low-mistake environment, all strategies that can forgive the other player's mistakes become successful and dominate the population. This safe environment allows even the always cooperate strategy to persist. This leads to high collaboration with a maximum average score per game of almost 2.0.
10% Mistake: If more mistakes are made, not all types of forgiveness are effective. Many mistakes can cause the cooperation between Forgiving Grudger and Forgiving TitForTat (3,1,0) players to break down. Only the most forgiving TitForTat player of type (2,1,0) is still able to collaborate effectively.
20% Mistake: A higher mistake environment thus requires more forgiveness, but there is a limit. If the mistake percentage is too high, it allows cheaters to exploit the high forgiveness of others. In this simulation we can indeed see that cheating becomes the dominant strategy and the average reward drops to 1.0.
Experiment 3: Exploiting Forgiveness
In environments where mistakes can be made, being forgiving is rewarded. However, forgiveness can be exploited by making intentional "mistakes". In this simulation we introduce sneaky strategies that deviate from their regular strategy a certain percentage of the time and cheat.
1% Mistake:
From this simulation we can see and interesting cyclic behaviour of the forgiving, non-forgiving and bertayel strategies. We can see the green
forgiving strategies are often the dominant strategies, but as the environment becomes forgiving, bertayel pays off and the red
types dominate. This leads players to adapt to less forgiving strategies like the blue TitforTat strategy, which decentives
cheating and causes the forgiving startegy to dominate once again.
When the forgiving types dominate the population cooperation is the highest, so one would think that the more forgiving you are the better! But no, actually it is the most forgiving types that causes betrayel to florish. If cheating is never punished more people will cheat.
10% Mistake: In an environment where more mistakes are made, sneaky behavior is rewarded. This causes players with more forgiving strategies to either be more sneaky or less forgiving. This behavior hurts effective cooperation and the average reward cycles around 1.6.
20% Mistake: Again, when mistakes are too high, effective collaboration breaks down and the average reward drops to 1.0.
Conclusion
The experiments show us that in an environment without mistakes, the TitForTat and Grudger strategies are optimal and can't be exploited. However, when people can make mistakes, non-forgiving strategies cause cooperation to break down. Forgiving strategies prove useful in environments where honest mistakes are often made. Adding the possibility for players to deviate from the strategy and cheat reveals that excessive forgiveness can be exploited, leading the dominant strategies to either cheat more or be less forgiving.
As much as I would like to assign profound meaning to the findings of these experiments, the outcome of these experiments is sensitive to how the environment is set up, how we do selection, and players pay-off. We should thus be careful to draw far-reaching conclusions from these types of game theoretic simulations. As we have seen from these simulations, adding extra realism such as the possibility for mistakes and different player behaviors greatly impacts what the dominant strategies look like.