With the rapid expansion of smart applications and services, the Internet of Things (IoT) plays a vital role in enabling changes in the usage of information and communication technology. Advanced Metering Infrastructure (AMI) is the key component of the smart grid which often meets privacy and security threats from the routing layer. The Routing Protocol for Low-Power and Lossy Networks (RPL) is the prominent routing protocol for AMI networks, and securing the RPL is the cornerstone of all security controls in the AMI for successful deployment. Recently, IoT security systems have increasingly adopted privacy-preserving Federated Learning (FL) that trains a local data model across multiple decentralized edge devices and takes global decisions without the need for global data sharing. Relying on a single learning model significantly degrades the FL performance due to the frequent transmission of weights and training on heterogeneous IoT networks. This paper proposes an Optimized FL model for Securing RPL (OFL-SRPL) over AMI devices. By reducing the size of the data transferred from the multiple local models to the server, the OFL-SRPL adopted the Particle Swam Optimization (PSO) algorithm to select the best local model from the Home Area Network (HAN) gateway and build the global model. The proposed OFL-SRPL applies an ensemble of sequential classifiers with its appropriate loss function to improve the decision-making quality and reduce the FL communication burden. Instead of transferring all the local model weights to the global model, the OFL-SRPL iteratively selects the best ensemble model with sequential classifies and only sends the weight of the best model to build the global model. The sequential classifier design improves the ownership weight and handles class imbalance issues. After taking an optimized global model decision, the devices execute ensemble classifiers successfully with appropriate learning parameters. Thus, the proposed OFL-SRPL achieves 10.6% higher detection accuracy than a centralized setup of the Long Short-Term Memory (LSTM) model without incurring multiple communication rounds like traditional FL.