Recommender systems are subject to well-known popularity bias issues, that is, they expose frequently rated items more in recommendation lists than less-rated ones. Such a problem could also have varying effects on users with different gender, age, or rating behavior, which significantly diminishes the users' overall satisfaction with recommendations. In this paper, we approach the problem from the view of user personalities for the first time and discover how users are inclined toward popular items based on their personality traits. More importantly, we analyze the potential unfairness concerns for users with different personalities, which the popularity bias of the recommenders might cause. To this end, we split users into groups of high, moderate, and low clusters in terms of each personality trait in the big-five factor model and investigate how the popularity bias impacts such groups differently by considering several criteria. The experiments conducted with 10 well-known algorithms of different kinds have concluded that less-extroverted people and users avoiding new experiences are exposed to more unfair recommendations regarding popularity, despite being the most significant contributors to the system. However, discrepancies in other qualities of the recommendations for these user characteristics, such as accuracy, diversity, and novelty, vary depending on the utilized algorithm.