The paper addresses the problem of tracking multiple targets with Doppler-only measurements in multi-sensor systems. It is well known that the observability of the target state measured using Doppler-only measurements is very poor, which makes it difficult to initialize the tracking target and produce the target trajectory in any tracking algorithm. Within the framework of random finite sets, we propose a novel constrained admissible region (CAR) based birth model that instantiates the birth distribution using Doppler-only measurements. By combining physics-based constraints in the unobservable subspace of the state space, the CAR based birth model can effectively reduce the ambiguity of the initial state. The CAR based birth model combines physics-based constraints in the unobservable subspace of the state space to reduce the ambiguity of the initial state. We implement the CAR based birth model with the generalized labeled multi-Bernoulli tracking filter to demonstrate the effectiveness of our proposed algorithm in Doppler-only tracking. The performance of the proposed approach is tested in two simulation scenarios in terms of the optimal subpattern assignment (OSPA) error, OSPA(2) error, and computing efficiency. The simulation results demonstrate the superiority of the proposed approach. Compared to the approach taken by the state-of-the-art methods, the proposed approach can at most reduce the OSPA error by 58.77%, reduce the OSPA(2) error by 43.51%, and increase the computing efficiency by 9.56 times in the first scenario. In the second scenario, the OSPA error is reduced by 62.80%, the OSPA(2) error is reduced by 43.65%, and the computing efficiency is increased by 2.61 times at most.