Recently, a dynamic adaptive queue management with random dropping (AQMRD) scheme has been developed to capture the time-dependent variation of average queue size by incorporating the rate of change of average queue size as a parameter. A major issue with AQMRD is the choice of parameters. In this paper, a novel online stochastic approximation based optimization scheme is proposed to dynamically tune the parameters of AQMRD and which is also applicable for other active queue management (AQM) algorithms. Our optimization scheme significantly improves the throughput, average queue size, and loss-rate in relation to other AQM schemes.