With the rapid proliferation of RFID-based applications, RFID tags have been deployed into pervasive spaces in increasingly large numbers, e.g., the shelves of super markets are filled with tag-labeled items. Conventional localization schemes usually leverage precise anchor nodes to help compute the position of objects. However, it is usually difficult to find or deploy enough anchor nodes for accurate localization. In this paper, we propose solutions to locate the mobile users based on imprecise anchors in RFID systems. A large number of tags with approximate locations are used as anchor nodes to compute the user's locations. We thus present a time-efficient localization scheme to continuously tracking the mobile users. Experimental results indicate that our solutions can accurately locate the mobile users in a real-time approach. The improved method's accuracy is more than 30% better than the base solution.