In this paper, we study the number of equilibria of the replicator-mutator dynamics for both deterministic and random multi-player two-strategy evolutionary games. For deterministic games, using Descartes' rule of signs, we provide a formula to compute the number of equilibria in multi-player games via the number of change of signs in the coefficients of a polynomial. For two-player social dilemmas (namely the Prisoner's Dilemma, Snow Drift, Stag Hunt and Harmony), we characterize (stable) equilibrium points and analytically calculate the probability of having a certain number of equilibria when the payoff entries are uniformly distributed. For multi-player random games whose pay-offs are independently distributed according to a normal distribution, by employing techniques from random polynomial theory, we compute the expected or average number of internal equilibria. In addition, we perform extensive simulations by sampling and averaging over a large number of possible payoff matrices to compare with and illustrate analytical results. Numerical simulations also suggest several interesting behaviours of the average number of equilibria when the number of players is sufficiently large or when the mutation is sufficiently small. In general, we observe that introducing mutation results in a larger average number of internal equilibria than when mutation is absent, implying that mutation leads to larger behavioural diversity in dynamical systems. Interestingly, this number is largest when mutation is rare rather than when it is frequent.