This paper introduces the concept of optimization equilibrium as an equivalently versatile definition of a generalized Nash equilibrium for multi-agent non-cooperative games. Through this modified definition of equilibrium, we draw precise connections between generalized Nash equilibria, feasibility for bilevel programming, the Nikaido-Isoda function, and classic arguments involving Lagrangian duality and social welfare maximization. Significantly, this is all in a general setting without the assumption of convexity. Along the way, we introduce the idea of minimum disequilibrium as a solution concept that reduces to traditional equilibrium when equilibrium exists. The connections with bilevel programming and related semi-infinite programming permit us to adapt global optimization methods for those classes of problems, such as constraint generation or cutting plane methods, to the problem of finding a minimum disequilibrium solution. We show that this method works, both theoretically and with a numerical example, even when the agents are modeled by mixed-integer programs.