Radio maps can be used for source localization, link performance prediction, and wireless relay planning. This paper constructs an air-to-ground radio map to predict the channel gain for each link that connects a ground terminal with a low altitude unmanned aerial vehicle (UAV). The challenge is the insufficiency of measurement samples for a radio map in full dimension, where each data point is 6-dimensional as the transmitter and the receiver each has three spatial degrees of freedom. Classical methods, such as k-nearest neighbor (KNN) and Kriging, may fail for insufficient data. This paper proposes to exploit the propagation property in the geometry of the environment to assist the radio map construction. Specifically, the radio map is built via reconstructing a virtual geometry environment. A multi-class virtual obstacle model embedded in a multi-degree channel model is developed, and a least squares problem is formulated to learn the virtual obstacle map and the model parameters. The paper investigates the partial quasiconvexity of the least squares problem, and based on that, an efficient radio map learning algorithm is developed. In addition, a data driven approach is employed to build a residual shadowing map to further improve the details of the constructed radio map. Our numerical results confirm that the proposed method significantly increases the prediction accuracy compared to a Kriging baseline, and reduces the required measurements by more than a half. When the constructed radio map is applied to received signal strength (RSS) based localization in a dense urban environment, substantial performance improvement is observed where a sub-20-meter accuracy is achieved.