Abstract: In game theoretic learning, e.g., for matrix games and population games, agents myopically adapt their strategies in reaction to the evolving strategies of other agents in an effort to maximize their own utilities. The resulting interactions can be represented as a dynamical system that maps agent observations to agent strategies. Well-known and widely studied examples of adaptation/learning rules include fictitious play, gradient play, regret minimization, and replicator dynamics.  In these examples, the associated learning rule has an induced dimensionality, or number of states, that is equal to the number of agent actions. As the terminology suggests, “higher-order” learning refers to learning rules that are not restricted in their dimensionality. Such learning rules introduce auxiliary states not included in their lower-order counterparts, while respecting the original informational structure of what is observed and known to each agent. 

This talk presents on overview of results that illustrate how higher-order learning can induce qualitative changes in long-run outcomes, including convergence to Nash equilibria not possible under lower-order dynamics (including uncoupled dynamics counterexamples and replicator dynamics for zero-sum games). A specific focus will be on higher-order “anticipatory” versions of lower-order learning rules, which appears to parallel optimistic versions of optimization algorithms. The talk concludes with an analysis framework for higher-order learning that exploits an implicit feedback structure in game-theoretic learning, where the learning dynamics are separated from the game specifics. In particular, the talk presents the concept of passivity from feedback control, its application to higher-order learning analysis, and connections to contractive/stable games.

Bio: Jeff S. Shamma is with the University of Illinois at Urbana-Champaign where he is the Department Head of Industrial and Enterprise Systems Engineering (ISE) and Jerry S. Dobrovolny Chair in ISE. His prior academic appointments include faculty positions at the King Abdullah University of Science and Technology (KAUST), where he is an Adjunct Professor of Electrical and Computer Engineering, and the Georgia Institute of Technology, where he was the Julian T. Hightower Chair in Systems and Controls. Jeff received a PhD in Systems Science and Engineering from MIT in 1988. He is a Fellow of IEEE and IFAC; a recipient of the IFAC High Impact Paper Award, AACC Donald P. Eckman Award, and NSF Young Investigator Award; and a past Distinguished Lecturer of the IEEE Control Systems Society. He has been a plenary or semi-plenary speaker at several conferences, including NeurIPS, World Congress of the Game Theory Society, IEEE Conference on Decision and Control, and the American Control Conference. Jeff is currently serving as the Editor-in-Chief for the IEEE Transactions on Control of Network Systems.