Extracellular matrix (ECM) structure and biochemistry provide cell-instructive cues that promote and regulate tissue growth, function, and repair. From a structural perspective, the ECM is a scaffold that guides the self-assembly of cells into distinct functional tissues. The ECM promotes the interaction between individual cells and between different cell types, and increases the strength and resilience of the tissue in mechanically dynamic environments. From a biochemical perspective, factors regulating cell-ECM adhesion have been described and diverse aspects of cell-ECM interactions in health and disease continue to be clarified. Natural ECMs therefore provide excellent design rules for tissue engineering scaffolds. The design of regenerative three-dimensional (3D) engineered scaffolds is informed by the target ECM structure, chemistry, and mechanics, to encourage cell infiltration and tissue genesis. This can be achieved using nanofibrous scaffolds composed of polymers that simultaneously recapitulate 3D ECM architecture, high-fidelity nanoscale topography, and bio-activity. Their high porosity, structural anisotropy, and bio-activity present unique advantages for engineering 3D anisotropic tissues. Here, we use the heart as a case study and examine the potential of ECM-inspired nanofibrous scaffolds for cardiac tissue engineering. We asked: Do we know enough to build a heart? To answer this question, we tabulated structural and functional properties of myocardial and valvular tissues for use as design criteria, reviewed nanofiber manufacturing platforms and assessed their capabilities to produce scaffolds that meet our design criteria. Our knowledge of the anatomy and physiology of the heart, as well as our ability to create synthetic ECM scaffolds have advanced to the point that valve replacement with nanofibrous scaffolds may be achieved in the short term, while myocardial repair requires further study in vitro and in vivo.