To improve the positioning accuracy of an inertial/geomagnetic integrated navigation algorithm, a combined navigation method based on matching strategy and hierarchical filtering is proposed. First, the PDA-ICCP geomagnetic matching algorithm is improved. On basis of evaluating the distribution of magnetic measurements, a number of controllable magnetic values are regenerated to participate in the geomagnetic matching algorithm (GMA). As a result, accuracy of the matching algorithm is ensured and its efficiency is improved. Secondly, the integrated navigation filter is designed based on the hierarchical filtering strategy, in which the navigation information of the geomagnetic matching module and inertial navigation module are respectively filtered and fused in the main filter. In this way, the shortcoming that GMA is unable to provide continuous and real-time navigation information is overcome. Meanwhile, precision of the inertial/geomagnetic integrated navigation algorithm is improved. Finally, the feasibility and validity of the proposed algorithm are verified by simulation and physical experiments. Compared with the integrated filtering algorithm which directly uses the error equation of inertial navigation system (INS) as the state equation, the proposed hierarchical filtering algorithm can achieve higher positioning precision.