Along with the advancement of online platforms and significant growth in Internet usage, various threats and cyber-attacks have been emerging and become more complicated and perilous in a day-by-day base. Anomaly-based intrusion detection systems (AIDSs) are lucrative techniques for dealing with cybercrimes. As a relief, AIDS can be equipped with artificial intelligence techniques to validate traffic contents and tackle diverse illicit activities. A variety of methods have been proposed in the literature in recent years. Nevertheless, several important challenges like high false alarm rates, antiquated datasets, imbalanced data, insufficient preprocessing, lack of optimal feature subset, and low detection accuracy in different types of attacks have still remained to be solved. In order to alleviate these shortcomings, in this research a novel intrusion detection system that efficiently detects various types of attacks is proposed. In preprocessing, Smote-Tomek link algorithm is utilized to create balanced classes and produce a standard CICIDS dataset. The proposed system is based on gray wolf and Hunger Games Search (HGS) meta-heuristic algorithms to select feature subsets and detect different attacks such as distributed denial of services, Brute force, Infiltration, Botnet, and Port Scan. Also, to improve exploration and exploitation and boost the convergence speed, genetic algorithm operators are combined with standard algorithms. Using the proposed feature selection technique, more than 80 percent of irrelevant features are removed from the dataset. The behavior of the network is modeled using nonlinear quadratic regression and optimized utilizing the proposed hybrid HGS algorithm. The results show the superior performance of the hybrid algorithm of HGS compared to the baseline algorithms and the well-known research. As shown in the analogy, the proposed model obtained an average test accuracy rate of 99.17%, which has better performance than the baseline algorithm with 94.61% average accuracy.