Integrating celestial navigation system (CNS) and inertial navigation system (INS) is an effective way to realize autonomous navigation for hypersonic cruise vehicles. In multi-stars synchronous observation, the error levels of starlight observations are not only changing with time, but also different from star to star, which is ignored by classic INS/CNS integrated navigation. In this paper, a new tightly-coupled INS/CNS integrated navigation algorithm with weighted multistars observations is proposed. Different from the classic method, tightly-coupled model provides a novel framework for INS/CNS integration, which enables direct fusion of INS measurements with original CNS observations. Meanwhile, star subsets traversal strategy is designed to evaluate the error level of each observation. Then adaptive filter which can fuse multi-star observations with INS according to evaluated error levels is researched. The proposed new algorithm shows significant improvement in navigation accuracy than the classic method in simulation.