Flywheels play a critical role as core components in satellite attitude control systems. However, their high-speed rotation inevitably generates vibrations that have a detrimental impact on the in-orbit imaging capabilities of high-precision remote sensing payloads. This study focuses on the passive vibration isolation design of satellite flywheels. The flywheel-mounted structural plate and flywheel vibration isolation platform are considered as a whole system (termed a plate-isolator system). In this system, the structural plate is treated as an elastomer. By simplifying the plate-isolator system as a 2-degree-of-freedom vibration system, it becomes evident that obtaining an ideal vibration isolation effect through the optimization of the flywheel vibration isolation platform (FVIP) alone is difficult. In order to enhance the passive vibration isolation effect for satellite flywheels, this study introduces the concept of an elastic boundary applied to the flywheel-mounted structural plate, thus treating the elastic boundary as a design factor. Consequently, the plate-isolator system can be simplified as a 3-stage vibration isolation system. The optimization of the elastic boundary condition of the structural plate is performed using the kinetic model of the simplified 3-stage system. The vibration isolation effect of the plate-isolator system with an elastic boundary is further confirmed through finite element simulation. The calculation results demonstrate that, after establishing a reasonable elastic boundary for the satellite structural plate, the overall vibration/force transmission rate of the plate-isolator system becomes similar to that of a single-degree-of-freedom dynamic system. Finally, the proposed concept is validated through kinetic response analysis of a cube satellite. The results reveal that the vibration amplitude of the satellite’s top and side structural plates can be effectively lowered if the elastic boundary condition is set for the flywheel-mounted bottom structural plate.