AI Pricing Collusion: Multi-Agent Reinforcement Learning Algorithms in Bertrand Competition

Abstract

As e-commerce and online shopping become more widespread, firms are starting to maximize profit by using artificial intelligence, or more specifically reinforcement learning, to price goods. Calvano et al. showed that in a simple market with multiple agents controlled by a reinforcement learning algorithm, the agents learn to collude above the competitive Nash equilibrium price, and that these agents even punish deviations from this collusion, unlike what we see when humans control pricing. I experimentally analyze these results and replicate them, noting slight discrepancies. Then I extend these results to a more practical and realistic setting using more complex reinforcement learning algorithms, finding that these algorithms collude more reliably and much faster at a higher price than the original, simple algorithm presented by Calvano et al. I then consider ways to mitigate this collusion; one by introducing a supervisor agent that changes demand resembling the Amazon "buy box" technique, and another by introducing mechanisms that force prices downward at the cost of market interference.