Recent years have witnessed a sharp increase of malicious apps that steal users' personal information. To address users' concerns about privacy risks and to comply with data protection laws, more and more apps are supplied with privacy policies written in natural language to help users understand an app's privacy practices. However, little is known whether these privacy policies are trustworthy or not. Questionable privacy policies may be prepared by careless app developers or someone with malicious intention. In this paper, we carry out a systematic study on privacy policy by proposing a novel approach to automatically identify five kinds of problems in privacy policy. After tackling several challenging issues, we implement the approach in a system, named PPChecker, and evaluate it with real apps and their privacy policies. The experimental results show that PPChecker can effectively identify questionable privacy policies with high precision. Applying PPChecker to 2,500 popular apps, we find that 1,850 apps (i.e., 74.0%) have at least one kind of problems. This study sheds light on the research of improving and regulating apps' privacy policies.