Microscopic and ultramicroscopic vascular sutures are indispensable in surgical procedures such as arm transplantation and finger reattachment. The state of the blood vessels after suturing, which may feature vascular patency, narrowness, and blockage, determines the success rate of the operation. If we can take advantage of the golden window of opportunity after blood vessel suture and before muscle tissue suture to achieve an accurate and objective assessment of blood vessel status, this will not only reduce medical costs but will also offer social benefits. Doppler optical coherence tomography enables the high-speed, high-resolution imaging of biological tissues, especially microscopic and ultramicroscopic blood vessels. By using Doppler optical coherence tomography to image the sutured blood vessels, a three-dimensional structure of the blood vessels and blood flow information can be obtained. By extracting the contour of the blood vessel wall and the contour of the blood flow area, the three-dimensional shape of the blood vessel can be reconstructed in three dimensions, providing parameter support for the assessment of blood vessel status. In this work, we propose a neural network-based multi-classification deep learning model that can automatically and simultaneously extract blood vessel boundaries from Doppler OCT vessel intensity images and the contours of blood flow regions from corresponding Doppler OCT vessel phase images. Compared to the traditional random walk segmentation algorithm and cascade neural network method, the proposed model can produce the vessel boundary from the intensity image and the lumen area boundary from the corresponding phase image simultaneously, achieving an average testing segmentation accuracy of 0.967 and taking, on average, 0.63 s. This method can realize system integration more easily and has great potential for clinical evaluations. It is expected to be applied to the evaluation of microscopic and ultramicroscopic vascular status in microvascular anastomosis.