Industry 4.0 drives exponential growth in the amount of operational data collected in factories. These data are commonly distributed and stored in different business units or cooperative companies. Such data-rich environments increase the likelihood of cyber attacks, privacy breaches, and security violations. Also, this poses significant challenges on developing machine learning models on sensitive data that are distributed among different business units. To fill this gap, this paper presents a novel privacy-preserving framework to enable federated learning on siloed and encrypted data for smart manufacturing. Specifically, we leverage fully homomorphic encryption (FHE) to allow for computation on ciphertexts and generate encrypted results which, when decrypted, match the results of mathematical operations performed on the plaintexts. Multi-layer encryption and privacy protection reduce the likelihood of data breaches while maintaining the prediction performance of machine learning models. Experimental results in real-world case studies show that the proposed framework yields superior performance to reduce the risk of cyber attacks and harness siloed data for smart manufacturing.