Additive manufacturing has become an important tool for fabricating advanced systems and devices for visible nanophotonics. However, the lack of simulation and optimization methods taking into account the essential physics of the optimization process leads to barriers for greater adoption. This issue can often result in sub-optimal optical responses in fabricated devices on both local and global scales. We propose that physics-informed design and optimization methods, and in particular physics-informed machine learning, are particularly well-suited to overcome these challenges by incorporating known physics, constraints, and fabrication knowledge directly into the design framework.