The protein tyrosine phosphatase SHP2 mediates multiple signal transductions in various cellular pathways, controlled by a variety of upstream inputs. SHP2 dysregulation is causative of different types of cancers and developmental disorders, making it a promising drug target. However, how SHP2 is modulated by its different regulators remains largely unknown. Here, we use single-molecule fluorescence resonance energy transfer and molecular dynamics simulations to investigate this question. We identify a partially open, semiactive conformation of SHP2 that is intermediate between the known open and closed states. We further demonstrate a “multiple gear” regulatory mechanism, in which different activators (
e.g.,
insulin receptor substrate-1 and CagA), oncogenic mutations (
e.g.,
E76A), and allosteric inhibitors (
e.g.,
SHP099) can shift the equilibrium of the three conformational states and regulate SHP2 activity to different levels. Our work reveals the essential role of the intermediate state in fine-tuning the activity of SHP2, which may provide new opportunities for drug development for relevant cancers.