The efficient dissipation of heat has emerged as a crucial concern for modern electronic devices, given the continuous increase in their power density and consumption. Thus, the utilization of thermally conductive but electrically insulating silicone rubber composites as a thermal interface material has garnered significant interest. In this study, the effects of the filler volume fraction, filler orientation, layer volume fractions, layer configuration, and a number of layers on the thermal conductivity and electrical resistivity of silicone rubber composites were examined using a multiscale finite element modeling strategy. The results demonstrated that modification of the filler orientation can change the thermal conductivity by 28 and 21 times in the in-plane and through-thickness directions, respectively. The in-plane thermal conductivities of silicone rubber/boron nitride and silicone rubber/expanded graphite layers exhibit a percolation phenomenon at filler volume fractions of 35% and 30%, respectively. The electrical resistivity of the composite increases exponentially with a decrease in the number of layers.