Over recent years, the world has experienced explosive growth in the volume of textual data, which makes a manual analysis impossible. Machine learning techniques provided an effective solution to this problem. Due to its capacity to organize the huge and varied amounts of data, it offered valuable insights and it has become an emerging investigative field for the research community. Classification techniques are used to classify data into different classes according to desired criteria. By their simplicity, they give rise to a variety of applications: automated text categorization, opinion mining, and so forth. These processes go through three stages: text representation, features extraction, and the classification process; they still face many difficulties due both to the complex nature of text databases and to the high dimensionality of texts representations. This article presents a new classification approach that learns to classify texts from the most reliable features more accurately. The added advantage of the proposed approach is that it automatically classifies a text without necessarily processing all its features. The experimental results showed that this new classification by thresholds outperforms the state‐of‐the‐art methods. As a result, the obtained f‐measure on automatic text categorization was 95.06% while it is lower on opinion mining.