Integrated photonics offers the possibility of compact, low energy, bandwidth-dense interconnects for large port count spatial optical switches, facilitating flexible and energy efficient data movement in future data communications systems. To achieve widespread adoption, intimate integration with electronics has to be possible, requiring switch design using standard microelectronic foundry processes and available devices. We report on the feasibility of a switch fabric comprised of ubiquitous silicon photonic building blocks, opening the possibility to combine technologies, and materials towards a new path for switch fabric design. Rather than focus on integrating all devices on a single silicon chip die to achieve large port count optical switching, this work shifts the focus towards innovative packaging and integration schemes. In this work, we demonstrate 1 × 8 and 8 × 1 microring-based silicon photonic switch building blocks with software control, providing the feasibility of a full 8 × 8 architecture composed of silicon photonic building blocks. The proposed switch is fully non-blocking, has path-independent insertion loss, low crosstalk, and is straightforward to control. We further analyze this architecture and compare it with other common switching architectures for varying underlying technologies and radices, showing that the proposed architecture favorably scales to very large port counts when considering both crosstalk and architectural footprint. Separating a switch fabric into functional building blocks via multiple photonic integrated circuits offers the advantage of piece-wise manufacturing, packaging, and assembly, potentially reducing the number of optical I/O and electrical contacts on a single die.