A unique photonic crystal fiber with square and circular air holes (SC-PCF) is designed in this research. Three layers of circular air holes and two levels of square air holes make up the fiber cladding. The finite element approach is used to simulate the fiber construction, and numerical calculations are used to examine the transmission properties in the S+C+L band. The results reveal that the SC-PCF can sustain 86 Orbital Angular Momentum (OAM) modes in the wavelength range of 1400 nm to 1700 nm (300 nm), with an effective refractive index difference (ERID) of 5.88 × 10−3 between them, thus avoiding mode coupling. The mode purity of all modes is greater than 96% at 1550 nm, and the lowest dispersion and dispersion change are 4.939 ps/nm/km and 0.956 ps/nm/km, respectively. The confinement loss (CL) of all modes is lower than 10−9 dB/m, and the nonlinear coefficient (NC) is lower than 1.5 W−1·km−1 in the whole band. The proposed SC-PCF has important value in long-distance and large-capacity communication systems.