Stability analysis of a Filippov Gause predator-prey model with or without hunting cooperation among predators with respect to prey density

Abstract

This paper investigates two new Filippov-Gause predator-prey models that show the transition between individual and cooperative hunting dynamics among predators with respect to the critical prey population size. By using Filippov systems, this research offers a more realistic and complete representation of the nonlinear dynamics and discontinuities inherent in this complex ecological system so that the proposed models provide a more complete view of predator-prey interactions in different realistic environmental contexts. For the case where predators only cooperate in hunting when the prey population size is larger than its critical value, we could have at least one asymptotically stable limit cycle around a single positive inner equilibrium, or two limit cycles around the pseudo-stable equilibrium. However, in the case where predators only cooperate in hunting when the prey population size is less than its critical value, we could have one limit cycle around two inner equilibria and one pseudo-equilibrium that is unstable. In general, as the two proposed models differ in the presence of a sliding or escaping segment, the overall dynamics and bifurcation diagrams of both models change significantly.