Continued device scaling enables microprocessors and other systems-on-chip (SoCs) to increase their performance, functionality, and hence, complexity. Simultaneously, relentless scaling, if uncompensated, degrades the performance and signal integrity of on-chip metal interconnects. These systems have therefore become increasingly communications-limited. The communications-centric nature of future high performance computing devices demands a fundamental change in intra-and inter-chip interconnect technologies.Optical interconnect is a promising long term solution. However, while significant progress in optical signaling has been made in recent years, applying conventional packetswitching interconnect architecture to optical networks require repeated E/O and O/E conversions that significantly diminish the advantages of optical signaling. In this paper, we propose to leverage a suite of newly-developed or emerging devices, circuits, and optics technologies to build a fully distributed interconnect architecture based on free-space optics. With a complexity-effective communication support layer to manage occasional packet collisions, the interconnect avoids packet relay altogether, offers an ultra-low transmission latency and scalable bandwidth, and provides fresh opportunities for coherency substrate designs and optimizations.