Gravitational Search Algorithm (GSA) is a recent stochastic search algorithm that is inspired from the concepts of gravity rule and law of motion in physics. Despite its success and attractiveness, it has some coefficients and parameters that should be properly tuned to improve its performance. This paper studies the performance of GSA by varying the parameters that controls its gravitational force. Then a new differential mutation operator is proposed to enhance performance of GSA by accelerating its convergence. The proposed algorithm, namely DMGSA, is evaluated using 15 well-known benchmark functions from the special session of CEC2013 with different characteristics including randomly shifted optimum, rotation and non-separability. The results obviously confirms the performance achieved from the proposed mutation operator outperforms that from the attempts of parameter tuning in the original GSA. Lastly, DMGSA is applied for optimizing a small-scale gene regulatory network. The result demonstrates that its performance is highly competitive and clearly surpasses original GSA.