Argus-II is one of the most successful epiretinal implantation for providing visual acuity those who lost their vision sight due to retinitis pigmentosa (RP) problem. However, this model faces color recognition issue is observed from implanted patients. Hence, it arises whenever electrode fail to retain same electrical stimuli property during sensitivity color transition state is occurred (especially, blue and purple colors). To resolve this problem, a proper handling of electrical stimuli parameters (amplitude, frequency and pulse width) is required during patient under every visual impact is possible. Addition to this, the individual patient color sensation is recorded in the observation state and creates Argus-II dataset to train the machine learning algorithm for maintaining phosphene brightness through controlled generation of the electrical stimuli. Therefore, in this paper, an automatic recognition of color sensation with controlled phosphene brightness using pre-trained CNNs framework is proposed. The frequency modulated electrical stimulation of retina is purely influence by trained CNNs for adjusting amplitude that can retain maximum brightness along with clarity in the color sensation. The experimental results shows that the proposed system is achieved reasonable improvement in the transition color sensation as well as controlled brightness when compared with other existing systems.