In this paper, we target at cognitively detecting the presence of the primary user (PU) as well as recognizing PU's signal modulation. Since the existing modulation classification methods rely on fixed sensing period which may waste time when the modulations are easier to distinguish, we propose an automatic modulation classification (AMC) approach using likelihood-based (LB) and feature-based (FB) sequential detection methods, where SU calculates the likelihood ratio (LLR) sequentially to determine whether or not to stop listening. Referring to asymptotic analysis of the sequential methods, we formulate an optimization problem and derive the minimum sensing time under a constrained misclassification rate. Simulation results demonstrate that both LB and FB methods could significantly reduce the sensing time compared to fixed sensing period method.