Can AI Agents Learn to Collude? Emergence, Detection, and Mitigation
Project Description
Reinforcement learning (RL) agents are increasingly deployed in competitive markets such as electricity trading, e-commerce, and financial auctions. While these agents are designed to optimize individual payoffs, repeated interactions may result in tacit collusion, where agents implicitly learn to coordinate strategies without explicit communications, posing serious risks to market efficiency and consumer welfare.
This project explores the conditions under which collusive behaviors may emerge from RL agents and develops interventions to prevent them. We will (1) identify structural and algorithmic drivers of collusion in repeated stochastic games, and (2) design detection and mitigation strategies, such as reward perturbation, policy regularization, and Bayesian persuasion, to break collusive dynamics. Applications include electricity markets, where algorithmic bidding may distort prices and allocations.
This research will lay theoretical foundations at the intersection of control, optimization, and algorithmic game theory, and inform market design and regulatory safeguards for AI-powered marketplaces.
Start Date
Summer 2026
Postdoc Qualifications
• Ph.D. in EE, IE/OR, CS, or Applied Math
• Background in reinforcement learning, stochastic control, game theory, or market design
• Experience with multi-agent systems and/or auction theory is preferred
Co-advisors
Dr. Vijay Gupta, Elmore Professor
Elmore Family School of Electrical and Computer Engineering
gupta869@purdue.edu
Dr. Andrew L. Liu, Associate Professor
Edwardson School of Industrial Engineering
andrewliu@purdue.edu
Bibliography
|
[1] Shivam Bajaj, Pranoy Das, Yevgeniy Vorobeychik, and Vijay Gupta. Rationality of learning algorithms in repeated normal-form games. IEEE Control Systems Letters, 2024. [2] Feng, Chen, and Andrew L. Liu. "Peer-to-peer energy trading of solar and energy storage: A networked multiagent reinforcement learning approach." Applied Energy 383 (2025): 125283. [3] Liu, Andrew L., and Benjamin F. Hobbs. "Tacit collusion games in pool-based electricity markets under transmission constraints." Mathematical Programming 140, no. 2 (2013): 351-379. [4] Bhaskar Vundurthy, Aris Kanellopoulos, Vijay Gupta, and Kyriakos G Vamvoudakis. Intelligent players in a fictitious play framework. IEEE Transactions on Automatic Control, 69(1):479– 486, 2023. [5] Zhao, Zibo, Chen Feng, and Andrew L. Liu. "Comparisons of auction designs through multiagent learning in peer-to-peer energy trading." IEEE Transactions on Smart Grid 14, no. 1 (2022): 593-605. |