A convolutional neural network (CNN) architecture, NutriFoodNet, enhanced through Particle Swarm Optimization (PSO) is suggested in this paper to optimize data augmentation parameters and key hyperparameters, specifically designed for food image recognition. Accurate food image classification plays a vital function in various applications, including nutrition management, dietary assessment, and healthcare, as it aids in the automated recognition and analysis of food items from images. The implementation aimed to improve classification accuracy on the Food101 dataset. Initially, the NutriFoodNet model achieved an accuracy of 97.3%. By applying PSO, the model's performance was further refined, resulting in an increased accuracy of 98.5%. This optimized system was benchmarked against state-of-the-art architectures, including ResNet-18, ResNet-50, and Inception V3, showcasing its exceptional performance. The proposed system highlights the efficiency of PSO in fine-tuning augmentation parameters and CNN hyperparameters, leading to significant improvements in model accuracy for food image classification tasks. This advancement underscores the potential of enhanced food image classification systems in contributing to better dietary monitoring and healthcare outcomes.