High-precision indoor localization plays a vital role in various places. In recent years, visual inertial odometry (VIO) system has achieved outstanding progress in the field of indoor localization. However, it is easily affected by poor lighting and featureless environments. For this problem, we propose an indoor localization algorithm based on VIO system and three-dimensional (3D) map matching. The 3D map matching is to add height matching on the basis of previous two-dimensional (2D) matching so that the algorithm has more universal applicability. Firstly, the conditional random field model is established. Secondly, an indoor three-dimensional digital map is used as a priori information. Thirdly, the pose and position information output by the VIO system are used as the observation information of the conditional random field (CRF). Finally, the optimal states sequence is obtained and employed as the feedback information to correct the trajectory of VIO system. Experimental results show that our algorithm can effectively improve the positioning accuracy of VIO system in the indoor area of poor lighting and featureless.