Canna edulis Ker-Gawl is a versatile crop that integrates food, energy, and feed in China. The rhizome is a traditional Chinese medicine with a long history. However, there are few studies on chemical components. Crude plant extracts are a complex mixture of several biologically active secondary metabolites. Therefore, rapid and accurate identification and quantification are critical in phytochemical analysis. An efficient approach based on ultrahigh-performance liquid chromatography coupled with Q-Exactive Orbitrap tandem mass spectrometry (UHPLC Q-Exactive Orbitrap MS) was used to analyze the chemical components systematically. We used retention time, accurate molecular weight, parent peaks, fragment peaks, fragmentation characteristics, comparative analysis with the literature, reference standards, and standard databases, including ChemSpider, mzVault, MassBank, and mzCloud. A total of 54 chemical constituents were identified from the methanol extract of the rhizome, including 36 organic acids (with six phenolic acids), three phenols, one sugar (and its derivatives), one coumarin, one triterpenoid, ten N-containing organic compounds (including seven amino acids), and two other compounds. Moreover, three compounds were quantitatively analyzed and a methodological study of the three components was carried out. The established method provided satisfactory precision and accuracy, acceptable recovery, good linearity, and a reasonable detection limit. The UHPLC Q-Exactive Orbitrap MS method was used for the first time to accurately, quickly, and systematically characterize the compounds of the rhizome, revealing the pharmacodynamic substances and providing a theoretical basis for its quality control, in-depth research, and development.