Protein tyrosine phosphatase SHP2 functions as a key regulator of cell cycle control, and activating mutations cause several cancers. Here, we dissect the energy landscape of wild-type SHP2 and the oncogenic mutation E76K. NMR spectroscopy and X-ray crystallography reveal that wild-type SHP2 exchanges between closed, inactive and open, active conformations. E76K mutation shifts this equilibrium toward the open state. The previously unknown open conformation is characterized, including the active-site WPD loop in the inward and outward conformations. Binding of the allosteric inhibitor SHP099 to E76K mutant, despite much weaker, results in an identical structure as the wild-type complex. A conformational selection to the closed state reduces drug affinity which, combined with E76K’s much higher activity, demands significantly greater SHP099 concentrations to restore wild-type  activity levels. The differences in structural ensembles and drug-binding kinetics of cancer-associated SHP2 forms may stimulate innovative ideas for developing more potent inhibitors for activated SHP2 mutants.