With the rapid growth in intelligent metasurfaces in the recent years, deep learning has attracted attention to transform the ways in which metasurfaces are simulated and designed. The unique advantages of deep learning lie in the powerful data‐driven modality, which allows a computational model to learn useful information using hierarchically structured layers. Among the various successful examples, there are forward and inverse designs. However, such designs are inherently data‐hungry. Thus, the data utilization efficiency must be maximized, and green metasurface design must be achieved. Here, the authors propose heterogeneous transfer learning to allow transferrable and data‐efficient metasurface design. The key to this method is a flexible network framework, which integrates feature augmentation and dimensionality reduction. The concept is demonstrated through three scenarios, i.e., metasurfaces with different parameterizations, different physical sizes, and completely different geometries, where the relative error reduction reaches up to 50%. Furthermore, an inverse metasurface design is proposed, which combines the forward predicted network and heuristic algorithm. This work considerably reduces the workload on data collection and overcomes the limitation that previous works only focused on fixed physical structures. The authors have also envisioned a “global metasurface gene bank,” in which researchers can freely “withdraw and save data” for various applications.