Harris Hawks Optimization (HHO) algorithm was proposed recently under the metaheuristic algorithms, which can fix many problems in various domains. However, it needs to improve in local search, which may lead to a loss of diversity, stuck in a local minimum, which procures premature convergence. Two steps have been introduced in this paper to avoid these issues. Firstly, integrating Opposition-based learning (OBL) with HHO accelerates the search process and enhances the choice of better solutions, namely OHHO. Then, using reinforcement learning to enhance the research technique of OHHO, called RLOHHO. CEC 2015 and CEC 2017 benchmark functions and real engineering problems are utilized to evaluate the efficiency. Finally, the proposed versions of HHO are compared with efficient optimization algorithms. The experiment results illustrate that the RLOHHO's version achieved better solutions than the original HHO, OHHO, and other similar published algorithms in the literature.