Fine-scale wind field nowcasting is of great significance in air traffic management, power grid operation, and so on. In this article, an indirect wind field nowcasting scheme based on lidar observation is presented, which contains an encoderforecaster network based on the convolutional long short-term memory (ConvLSTM) with balanced structure and a mask branch. The proposed nowcasting network is trained and evaluated based on the lidar observations throughout 2020 at Hong Kong International Airport. Comprehensive comparison with nine methods including the widely used optical flow technique and classic neural network show the good performance of the new network. It can capture the spatio-temporal features in the lidar observations and obtain better nowcasting results up to 27 minutes with a resolution of 100 m. The nowcasting errors are smaller than the retrieval errors reported in recent literatures, demonstrating that the lidar observation nowcasting based on the new network can get fine-scale wind field nowcasting results with high efficiency.