According to the hypothesis of Maloney et al. (Cognition 114(2):293–297, 2010. https://doi.org/10.1016/j.cognition.2009.09.013), math anxiety is related to deficits in numerical magnitude processing, which in turn compromises the development of advanced math skills. Because previous studies on this topic are contradictory, which may be due to methodological differences in the measurement of numerical magnitude processing, we tested Maloney et al.’s hypothesis using different tasks and their indicators: numerical magnitude processing (symbolic and non-symbolic comparison tasks: accuracy, reaction time, numerical ratio, distance and size effects, and Weber fraction; number line estimation task: estimation error), math anxiety (combined scores of learning, testing, math problem solving, and general math anxiety), and math performance. The results of our study conducted on 119 young adults mostly support the hypothesis proposed by Maloney et al. that deficiency in symbolic magnitude processing is related to math anxiety, but the relationship between non-symbolic processes and math anxiety was opposite to the assumptions. Moreover, the results indicate that estimation processes (but not comparison processes) and math anxiety are related to math performance in adults. Finally, high math anxiety moderated the relationship between reaction time in the symbolic comparison task, reaction time in the non-symbolic comparison task, numerical ratio effect in the symbolic comparison task, and math performance. Because the results of the joint effect of numerical magnitude processing and math anxiety on math performance were inconsistent, this part of the hypothesis is called into question.