As a promising new dyeing process without using water, the non-aqueous medium dyeing of polyester has attracted people’s attention and some progress has been made in related research. However, the oligomers of polyester fiber can affect the dyeing of polyester during the use of a silicone waterless dyeing system. Based on this point, the oligomer problem in the silicone waterless dyeing system was investigated. The oligomers of some different types of polyester were extracted by solvent extraction. A treatment method with little influence on the fiber was used to reduce the oligomer content in polyester. The improvement of the dyeing effect of polyester after treatment in silicone medium was studied, and the influence of the oligomer on polyester dyeing was also analyzed. For the dyeing of disperse blue 366, the dye exhaustion was increased by 3.25–3.71%, and the color depth of the dyed sample was increased by 6–13%. Moreover, the colorfastness to rubbing was also improved. In the comparison, the changes in thermal properties and crystallization properties of polyester were tested by differential scanning calorimetry (DSC), dynamic mechanical analysis (DMA), and X-ray diffraction analysis (XRD). The results showed that the thermal and crystalline properties of polyester fiber were not changed before and after solvent extraction.