Forecasting the correct stock price is intriguing and difficult for investors due to its irregular, inherent dynamics, and tricky nature. Convolutional neural networks (CNN) have impressive performance in forecasting stock prices. One of the most crucial tasks when training a CNN on a stock dataset is identifying the optimal hyperparameter that increases accuracy. In this research, we propose the use of the Firefly algorithm to optimize CNN hyperparameters. The hyperparameters for CNN were tuned with the help of Random Search (RS), Particle Swarm Optimization (PSO), and Firefly (FF) algorithms on different epochs, and CNN is trained on selected hyperparameters. Different evaluation metrics are calculated for training and testing datasets. The experimental finding demonstrates that the FF method finds the ideal parameter with a minimal number of fireflies and epochs. The objective function of the optimization technique is to reduce MSE. The PSO method delivers good results with increasing particle counts, while the FF method gives good results with fewer fireflies. In comparison with PSO, the MSE of the FF approach converges with increasing epoch.