Glutamic acid is an abundant amino acid that lends a characteristic umami taste to foods. In fermented foods, glutamic acid can be found as a free amino acid formed by proteolysis or as a non-proteolytic derivative formed by microorganisms. The aim of the present study was to identify different structures of glutamic acid derivatives in a typical fermented protein-based food product, soy sauce. An acidic fraction was prepared with anion-exchange solid-phase extraction (SPE) and analyzed by UPLC/MS/MS and UPLC/TOF-MS. α-Glutamyl, γ-glutamyl, and pyroglutamyl dipeptides, as well as lactoyl amino acids, were identified in the acidic fraction of soy sauce. They were chemically synthesized for confirmation of their occurrence and quantified in the selected reaction monitoring (SRM) mode. Pyroglutamyl dipeptides accounted for 770 mg/kg of soy sauce, followed by lactoyl amino acids (135 mg/kg) and γ-glutamyl dipeptides (70 mg/kg). In addition, N-succinoylglutamic acid was identified for the first time in food as a minor compound in soy sauce (5 mg/kg).