In this paper, a novel secure communication scheme based on temporal spreading and dual-phase perturbation is proposed to improve security at the physical layer. For one thing, the traditional electro-optical feedback phase encryption system is significantly sophisticated by the introduction of additional synchronized chaotic phase encryption. Besides, more keys in this system are available, with the attendance of dispersion and optical chaos, the key space is notably enhanced to ∼236, resisting the eavesdropping attacks. After passing through a series of encryption devices, the information is ultimately displayed as a noise-like signal. A 32 Gb s−1 on-off keying signal is successfully transmitted over a 100 km standard single-mode fiber. Simulation results show that by selecting the appropriate system parameters, a bit error rate as low as 10−10 can be obtained for legitimate terminals, and the effect of different hardware parameter mismatches on the decryption performance is also investigated which indicates a certain tolerance range for the mismatch of each parameter. This paper provides a new idea to combine an electro-optical feedback system with optical chaos to establish optical secrecy communication.