We proposed a simple and efficient modular singlesource surface integral equation (SS-SIE) formulation for electromagnetic analysis of arbitrarily connected penetrable and perfectly electrical conductor (PEC) objects. In this formulation, a modular equivalent model for each penetrable object consisting of the composite structure is first independently constructed through replacing it by the background medium, no matter whether it is surrounded by the background medium, other media, or partially connected objects, and enforcing an equivalent electric current density on the boundary to remain fields in the exterior region unchanged. Then, by combining all the modular models and any possible PEC objects together, an equivalent model for the composite structure can be derived. The troublesome junction handling techniques are not needed and non-conformal meshes are intrinsically supported. The proposed SS-SIE formulation is simple to implement, efficient, and flexible, which shows significant performance improvement in terms of CPU time compared with the original SS-SIE formulation and the Poggio-Miller-Chang-Harrington-Wu-Tsai (PMCHWT) formulation. Several numerical examples including the coated dielectric cuboid, the large lossy objects, the planar layered dielectric structure, and the partially connected dielectric and PEC structure are carried out to validate its accuracy, efficiency and robustness.