Interference cancellation is one of the important issues in Heterogeneous networks (HetNets), due to the density of the network. In this paper, we investigate the problem of interference cancellation and resource allocation with the Q-learning approach. We consider the Inter-interference and Intra-interference between the femtocell and macro cells in the uplink scenario. With the aim of maximizing the QoS of the macro user equipment (MUE) and minimizing the interference of the MUE, a new reward function was proposed. The simulation results show the improvement of the proposed algorithm in two strong interference and increasing distance scenarios.