Single-pixel imaging is advancing rapidly in complex-amplitude imaging. However, reconstructing high-quality images demands significant acquisition and heavy computation, making the entire imaging process time-consuming. Here we propose what we believe to be a novel single-pixel complex-amplitude imaging (SCI) scheme using a complex-valued convolutional neural network for image reconstruction. The proposed sheme does not need to pre-train on any labeled data, and can quickly reconstruct high-quality complex-amplitude images with the randomly initialized network only under the constraints of the physical model. Simulation and experimental results show that the proposed scheme is effective and feasible, and can achieve a good balance between efficiency and quality. We believe that this work provides a new image reconstruction framework for SCI, and paves the way for its practical applications.