While there has been incredible progress in convex and nonconvex minimization, many problems in machine learning are in need of efficient algorithms to solve min-max optimization problems. However, unlike minimization, where algorithms can always be shown to converge to some local minimum, there is no notion of local equilibrium in min-max optimization that exists for general nonconvex-nonconcave functions. We will present new notions of local equilibria that are guaranteed to exist, efficient algorithms to compute it, and implications to GANs.