In order to solve the problems of narrow object orientation and insufficient visual rendering ability of the optical communication network experimental system, the author proposes a multithreaded scheduling using virtual reality, and the experimental system design method of optical communication network with multichannel serial port design is proposed. The system builds the overall structure model of the optical communication network experimental system in the virtual reality environment, and uses the PCI bus technology to build the transmission channel model of the optical communication network, the basic entity object of the optical communication network experimental system is constructed, and the multithread scheduling method is used to process the local information of the experimental system. The result obtained is as follows: The experimental system of optical communication network designed by the author’s method, the optical communication transmission bit error rate is reduced by about 30%, the time overhead is saved by about 60%, and the memory overhead is saved by about 50%, which is superior. Using the author’s method, the maximum bit error rate is about 22%, while the maximum bit error rate of traditional method 1 is about 43%, and the maximum bit error rate of traditional method 2 is about 70%. It is proven that the visual simulation effect of the optical communication network experimental system is good, the reliability and stability of the optical communication network are improved, and the communication transmission error is reduced.