Aphantasia is a characteristic in which people with normal perception have difficulty constructing their imagination. Most previous studies have used the Vividness of Visual Imagery Questionnaire (VVIQ), with some using self-identification of the absence of visual imagery, but there is a discrepancy between the proportions of aphantasia in the population calculated by these two criteria. It is unclear why this difference exists and how many people actually cannot form imagery. Moreover, because visual imagery is mainly focused upon, other types of aphantasia, relating to multi-sensory imagery, have not been fully investigated. We conducted an online sampling with a large number of participants (N = 2,885) to compare the proportions of aphantasia calculated by these two visual criteria, obtaining data from the same participants, and investigate the cognitive profile of multi-sensory imagery. The participants completed the VVIQ and Questionnaire upon Mental Imagery (QMI) and self-identified an absence of visual imagery. The proportions were 3.67% under the VVIQ criteria (VVIQ ≤ 32) and 12.24% under the self-identification of the absence of visual imagery criteria, roughly replicating the proportions of previous reports. Combining these visual criteria, in the group for low VVIQ (VVIQ ≤ 32), some participants showed the absence of all sensory imagery, while others showed specifically an absence of visual imagery. Individuals with aphantasia, identified by visual criteria, may have been mixed with those experiencing multi-sensory aphantasia. Our present study indicates that visual criteria are not sufficient to define multiple types of aphantasia and proposes that evaluations with multi-sensory imagery may help further characterize aphantasia for other types of sensory modality.