In the modern information society, people are constantly exposed to stress due to complex work environments and various interpersonal relationships. Aromatherapy is attracting attention as one of the methods for relieving stress using aroma. A method to quantitatively evaluate such an effect is necessary to clarify the effect of aroma on the human psychological state. In this study, we propose a method of using two biological indexes, electroencephalogram (EEG) and heart rate variability (HRV), to evaluate human psychological states during the inhalation of aroma. The purpose is to investigate the relationship between biological indexes and the psychological effect of aromas. First, we conducted an aroma presentation experiment using seven different olfactory stimuli while collecting data from EEG and pulse sensors. Next, we extracted the EEG and HRV indexes from the experimental data and analyzed them with respect to the olfactory stimuli. Our study found that olfactory stimuli have a strong effect on psychological states during aroma stimuli and that the human response to olfactory stimuli is immediate but gradually adapts to a more neutral state. The EEG and HRV indexes showed significant differences between aromas and unpleasant odors especially for male participants in their 20–30s, while the delta wave and RMSSD indexes showed potential for generalizing the method to evaluate psychological states influenced by olfactory stimuli across genders and generations. The results suggest the possibility of using EEG and HRV indexes to evaluate psychological states toward olfactory stimuli such as aroma. In addition, we visualized the psychological states affected by the olfactory stimuli on an emotion map, suggesting an appropriate range of EEG frequency bands for evaluating psychological states applied to the olfactory stimuli. The novelty of this research lies in our proposed method to provide a more detailed picture of the psychological responses to olfactory stimuli using the integration of biological indexes and emotion map, which contributes to the areas such as marketing and product design by providing insights into the emotional responses of consumers to different olfactory products.